386848602024052120260618
1546-170X3052024MayNature medicineNat MedAI-enabled electrocardiography alert intervention and all-cause mortality: a pragmatic randomized clinical trial.146114701461-147010.1038/s41591-024-02961-4The early identification of vulnerable patients has the potential to improve outcomes but poses a substantial challenge in clinical practice. This study evaluated the ability of an artificial intelligence (AI)-enabled electrocardiogram (ECG) to identify hospitalized patients with a high risk of mortality in a multisite randomized controlled trial involving 39 physicians and 15,965 patients. The AI-ECG alert intervention included an AI report and warning messages delivered to the physicians, flagging patients predicted to be at high risk of mortality. The trial met its primary outcome, finding that implementation of the AI-ECG alert was associated with a significant reduction in all-cause mortality within 90 days: 3.6% patients in the intervention group died within 90 days, compared to 4.3% in the control group (4.3%) (hazard ratio (HR) = 0.83, 95% confidence interval (CI) = 0.70-0.99). A prespecified analysis showed that reduction in all-cause mortality associated with the AI-ECG alert was observed primarily in patients with high-risk ECGs (HR = 0.69, 95% CI = 0.53-0.90). In analyses of secondary outcomes, patients in the intervention group with high-risk ECGs received increased levels of intensive care compared to the control group; for the high-risk ECG group of patients, implementation of the AI-ECG alert was associated with a significant reduction in the risk of cardiac death (0.2% in the intervention arm versus 2.4% in the control arm, HR = 0.07, 95% CI = 0.01-0.56). While the precise means by which implementation of the AI-ECG alert led to decreased mortality are to be fully elucidated, these results indicate that such implementation assists in the detection of high-risk patients, prompting timely clinical care and reducing mortality. ClinicalTrials.gov registration: NCT05118035 .© 2024. The Author(s), under exclusive licence to Springer Nature America, Inc.LinChin-ShengCSDivision of Cardiology, Department of Internal Medicine, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China.Medical Technology Education Center, School of Medicine, National Defense Medical Center, Taipei, Taiwan, Republic of China.LiuWei-TingWT0000-0002-3817-4079Division of Cardiology, Department of Internal Medicine, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China.TsaiDung-JangDJ0000-0002-5531-0810Medical Technology Education Center, School of Medicine, National Defense Medical Center, Taipei, Taiwan, Republic of China.Department of Artificial Intelligence and Internet of Things, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China.Department of Statistics and Information Science, Fu Jen Catholic University, New Taipei City, Taiwan, Republic of China.LouYu-ShengYSDepartment of Artificial Intelligence and Internet of Things, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China.ChangChiao-HsiangCHDivision of Cardiology, Department of Internal Medicine, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China.LeeChiao-ChinCCDivision of Cardiology, Department of Internal Medicine, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China.FangWen-HuiWHDepartment of Artificial Intelligence and Internet of Things, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China.Department of Family and Community Medicine, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China.WangChih-ChiaCCDepartment of Family and Community Medicine, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China.ChenYen-YuanYYDepartment and Graduate Institute of Medical Education and Bioethics, National Taiwan University College of Medicine, Taipei, Taiwan, Republic of China.LinWei-ShiangWSDivision of Cardiology, Department of Internal Medicine, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China.ChengCheng-ChungCCDivision of Cardiology, Department of Internal Medicine, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China.LeeChia-ChengCCDepartment of Medical Informatics, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China.Division of Colorectal Surgery, Department of Surgery, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China.WangChih-HungCH0000-0001-5058-4356Department of Otolaryngology-Head and Neck Surgery, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China.Graduate Institute of Medical Sciences, National Defense Medical Center, Taipei, Taiwan, Republic of China.TsaiChien-SungCSDivision of Cardiovascular Surgery, Department of Surgery, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China.LinShih-HuaSH0000-0001-6152-4885Division of Nephrology, Department of Internal Medicine, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China.LinChinC0000-0003-2337-2096Medical Technology Education Center, School of Medicine, National Defense Medical Center, Taipei, Taiwan, Republic of China. xup6fup@mail.ndmctsgh.edu.tw.Department of Artificial Intelligence and Internet of Things, Tri-Service General Hospital, National Defense Medical Center, Taipei, Taiwan, Republic of China. xup6fup@mail.ndmctsgh.edu.tw.School of Public Health, National Defense Medical Center, Taipei, Taiwan, Republic of China. xup6fup@mail.ndmctsgh.edu.tw.Graduate Institute of Aerospace and Undersea Medicine, National Defense Medical Center, Taipei, Taiwan, Republic of China. xup6fup@mail.ndmctsgh.edu.tw.engClinicalTrials.govNCT05118035Journal ArticleMulticenter StudyRandomized Controlled TrialPragmatic Clinical TrialResearch Support, Non-U.S. Gov't20240429
United StatesNat Med95020151078-8956IMAgedFemaleHumansMaleMiddle AgedArtificial IntelligenceElectrocardiography202341620243292024522152202443004220244292340ppublish3868486010.1038/s41591-024-02961-410.1038/s41591-024-02961-4Adhikari, N. K., Fowler, R. A., Bhagwanjee, S. & Rubenfeld, G. D. Critical care and the global burden of critical illness in adults. Lancet 376, 1339–1346 (2010).10.1016/S0140-6736(10)60446-1209342127136988Pronovost, P. J. et al. Physician staffing patterns and clinical outcomes in critically ill patients: a systematic review. JAMA 288, 2151–2162 (2002).10.1001/jama.288.17.215112413375Chalfin, D. B., Trzeciak, S., Likourezos, A., Baumann, B. M. & Dellinger, R. P. Impact of delayed transfer of critically ill patients from the emergency department to the intensive care unit. Crit. Care Med. 35, 1477–1483 (2007).10.1097/01.CCM.0000266585.74905.5A17440421Hodgetts, T. J. et al. Incidence, location and reasons for avoidable in-hospital cardiac arrest in a district general hospital. Resuscitation 54, 115–123 (2002).10.1016/S0300-9572(02)00098-912161290Jones, D. A., DeVita, M. A. & Bellomo, R. Rapid-response teams. N. Engl. J. Med. 365, 139–146 (2011).10.1056/NEJMra091092621751906Goldstein, B. A., Navar, A. M., Pencina, M. J. & Ioannidis, J. P. A. Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review. J. Am. Med. Inform. Assoc. 24, 198–208 (2017).10.1093/jamia/ocw04227189013Smith, G. B., Prytherch, D. R., Schmidt, P. E., Featherstone, P. I. & Higgins, B. A review, and performance evaluation, of single-parameter ‘track and trigger’ systems. Resuscitation 79, 11–21 (2008).10.1016/j.resuscitation.2008.05.00418620794Smith, G. B., Prytherch, D. R., Meredith, P., Schmidt, P. E. & Featherstone, P. I. The ability of the national early warning score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation 84, 465–470 (2013).10.1016/j.resuscitation.2012.12.01623295778Subbe, C. P., Kruger, M., Rutherford, P. & Gemmel, L. Validation of a modified Early Warning Score in medical admissions. QJM 94, 521–526 (2001).10.1093/qjmed/94.10.52111588210Mann, K. D. et al. Predicting patient deterioration: a review of tools in the digital hospital setting. J. Med. Internet Res. 23, e28209 (2021).10.2196/28209345910178517822Romero-Brufau, S. et al. Widely used track and trigger scores: are they ready for automation in practice? Resuscitation 85, 549–552 (2014).10.1016/j.resuscitation.2013.12.01724412159Ludikhuize, J. et al. Standardized measurement of the Modified Early Warning Score results in enhanced implementation of a rapid response system: a quasi-experimental study. Resuscitation 85, 676–682 (2014).10.1016/j.resuscitation.2014.02.00924561029Chan, P. S., Jain, R., Nallmothu, B. K., Berg, R. A. & Sasson, C. Rapid response teams: a systematic review and meta-analysis. Arch. Intern. Med. 170, 18–26 (2010).10.1001/archinternmed.2009.42420065195Tsai, D.-J. et al. Mortality risk prediction of the electrocardiogram as an informative indicator of cardiovascular diseases. Digit. Health 9, 20552076231187247 (2023).10.1177/205520762311872473744878110336769Raghunath, S. et al. Prediction of mortality from 12-lead electrocardiogram voltage data using a deep neural network. Nat. Med. 26, 886–891 (2020).10.1038/s41591-020-0870-z32393799Plana, D. et al. Randomized clinical trials of machine learning interventions in health care: a systematic review. JAMA Netw. Open 5, e2233946 (2022).10.1001/jamanetworkopen.2022.33946361736329523495Attia, Z. I. et al. Screening for cardiac contractile dysfunction using an artificial intelligence-enabled electrocardiogram. Nat. Med. 25, 70–74 (2019).10.1038/s41591-018-0240-230617318Attia, Z. I. et al. An artificial intelligence-enabled ECG algorithm for the identification of patients with atrial fibrillation during sinus rhythm: a retrospective analysis of outcome prediction. Lancet 394, 861–867 (2019).10.1016/S0140-6736(19)31721-031378392Downey, C. L., Tahir, W., Randell, R., Brown, J. M. & Jayne, D. G. Strengths and limitations of early warning scores: a systematic review and narrative synthesis. Int. J. Nurs. Stud. 76, 106–119 (2017).10.1016/j.ijnurstu.2017.09.00328950188van der Sijs, H., Aarts, J., Vulto, A. & Berg, M. Overriding of drug safety alerts in computerized physician order entry. J. Am. Med. Inform. Assoc. 13, 138–147 (2006).10.1197/jamia.M1809163573581447540Bedoya, A. D. et al. Minimal impact of implemented Early Warning Score and best practice alert for patient deterioration. Crit. Care Med. 47, 49–55 (2019).10.1097/CCM.0000000000003439302472396298839Embi, P. J. & Leonard, A. C. Evaluating alert fatigue over time to EHR-based clinical trial alerts: findings from a randomized controlled study. J. Am. Med. Inform. Assoc. 19, e145–e148 (2012).10.1136/amiajnl-2011-000743225340813392862Lyons, P. G., Edelson, D. P. & Churpek, M. M. Rapid response systems. Resuscitation 128, 191–197 (2018).10.1016/j.resuscitation.2018.05.013297777406147149Iung, B. & Vahanian, A. Epidemiology of valvular heart disease in the adult. Nat. Rev. Cardiol. 8, 162–172 (2011).10.1038/nrcardio.2010.20221263455de Lemos, J. A., McGuire, D. K. & Drazner, M. H. B-type natriuretic peptide in cardiovascular disease. Lancet 362, 316–322 (2003).10.1016/S0140-6736(03)13976-112892964Kollef, M. H. et al. A randomized trial of real-time automated clinical deterioration alerts sent to a rapid response team. J. Hosp. Med. 9, 424–429 (2014).10.1002/jhm.219324706596Perkins, G. D., Temple, R. M. & George, R. Time to intervene: lessons from the NCEPOD report. Resuscitation 83, 1305–1306 (2012).10.1016/j.resuscitation.2012.08.33222955050Yao, X. et al. Artificial intelligence-enabled electrocardiograms for identification of patients with low ejection fraction: a pragmatic, randomized clinical trial. Nat. Med. 27, 815–819 (2021).10.1038/s41591-021-01335-433958795Austrian, J. et al. Applying A/B testing to clinical decision support: rapid randomized controlled trials. J. Med. Internet Res. 23, e16651 (2021).10.2196/16651338350358065554Horwitz, L. I., Kuznetsova, M. & Jones, S. A. Creating a learning health system through rapid-cycle, randomized testing. N. Engl. J. Med. 381, 1175–1179 (2019).10.1056/NEJMsb190085631532967Simon, G. E., Platt, R. & Hernandez, A. F. Evidence from pragmatic trials during routine care—slouching toward a learning health system. N. Engl. J. Med. 382, 1488–1491 (2020).10.1056/NEJMp191544832294344Liu, X., Cruz Rivera, S., Moher, D., Calvert, M. J. & Denniston, A. K. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Nat. Med. 26, 1364–1374 (2020).10.1038/s41591-020-1034-x329082837598943Crawford, M. H. et al. ACC/AHA Guidelines for Ambulatory Electrocardiography. A report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Committee to Revise the Guidelines for Ambulatory Electrocardiography). Developed in collaboration with the North American Society for Pacing and Electrophysiology. J. Am. Coll. Cardiol. 34, 912–948 (1999).10.1016/S0735-1097(99)00354-X10483977Attia, Z. I., Harmon, D. M., Behr, E. R. & Friedman, P. A. Application of artificial intelligence to the electrocardiogram. Eur. Heart J. 42, 4717–4730 (2021).10.1093/eurheartj/ehab64934534279Huang, P.-F., Kung, P.-T., Chou, W.-Y. & Tsai, W.-C. Characteristics and related factors of emergency department visits, readmission, and hospital transfers of inpatients under a DRG-based payment system: a nationwide cohort study. PLoS ONE 15, e0243373 (2020).10.1371/journal.pone.0243373332964137725315Ribeiro, A. L. P. et al. Sami-Trop: 12-lead ECG traces with age and mortality annotations. Zenodo https://doi.org/10.5281/zenodo.4905617 (2021).Ribeiro, A. H. et al. CODE-15%: a large scale annotated dataset of 12-lead ECGs. Zenodo https://doi.org/10.5281/zenodo.4916205 (2021).