The study was conducted as a retrospective non-randomized cohort study using an FDA-approved, AI-based computer-aided triage, and prioritization solution (Aidoc, Tel Aviv, Israel) approved for the detection of all types of intracranial hemorrhages in radiology. Prior to the AI solution implementation (pre-AI), the radiologist worklist would be reviewed according to a FIFO (first in, first out) methodology. The AI solution was implemented at our emergency department institution between February 2018 and September 2018. The system operates as follows: all relevant CT studies are automatically sent for AI analysis with no manual trigger. Upon detection of suspected positive ICH findings, the AI solution delivers notifications directly to the radiologist workstation. The system was declared workflow-integrated and clinically ready once all the CT scanners and radiologist workstations were integrated. After the implementation, the radiologist worklist would receive notifications for scans with suspected positive ICH findings according to the AI algorithm, thus changing from the FIFO methodology to an urgency-cased methodology (post-AI). The description of the solution workflow is shown in Fig. 1.

The study was IRB approved with approval number 3532–16-SMC from September 7, 2020.

Data was retrieved from the Chameleon^© electronic health record (EHR) system in our hospital using the MDClone software—a data extraction and synthetization platform that provides patient-level index-organized event data (http://www.mdclone.com).

Data was extracted between the timeframes 1.1.2017–1.1.2018 (pre-AI) and 1.1.2019–1.1.2020 (post-AI). The AI solution was fully deployed in all the emergency department reading rooms served by the radiologists on call.

All patients 18 years and older that were admitted for the first time to our institution’s emergency department who underwent a non–contrast head computed tomography scan during the emergency department admission with a positive finding of any ICH type (subdural, epidural, subarachnoid, intraparenchymal, and intraventricular hemorrhage) on finalized radiologist report were included in the study population. The study population was split into two groups for the pre-AI and post-AI periods, respectively. Two additional comparable pathologies were included as controls: ischemic stroke (IS) and myocardial infarction (MI) during the same time periods. These served as internal controls to equalize the possible differences in healthcare protocols between the pre-AI and post-AI time periods. Both control groups included participants older than 18 years admitted for the first time to our institution’s emergency department. Ischemic stroke was defined as positive ischemic findings using computed tomography after a finalized radiologist report. Myocardial infarction was defined as a positive ischemic finding using troponin level change over time with supportive evidence (typical symptoms, suggestive electrocardiographic changes, imaging evidence of new myocardium abnormality). All included participants were discharged with the same primary diagnosis on admission. Acquisition and analysis of all the data described in Table 1 were done up to May 1, 2020, to enable a 120-day mortality analysis of participants admitted to the emergency department by December 31, 2019.

The confounding variables as well as clinical measures examined are shown in Table 1. Confounding variables included in the analysis were based on demographics, comorbidities, and medications: age, gender, smoking status, and complete blood count for white blood cells, hemoglobin, and platelet count, diabetes mellitus, hypertension, congestive heart failure, current use of anti-hypertensive agents, and anticoagulation and anti-aggregation agents (Table 3 in Supplements).

Emergency Severity Index (ESI) [21] was used to evaluate the participants’ clinical severity and therefore the emergency department resource utilization. An ESI score of 103 represents a median score (range of 101–105), and its percentile was used to compare the pre-AI and post-AI time periods.

Modified Rankin Scale for neurologic disability (mRS) was used for morbidity evaluation and was evaluated at admission (baseline) and discharge by a stroke neurologist according to standardized guidelines [21, 22].

Mortality was drawn from the hospital medical records software which is continuously updated by the Ministry of Health national mortality records.

Length of stay (LoS) was defined as the time (in days) from patient’s admission to the emergency room until complete discharge from the hospital or death.

Modified Rankin Scale was evaluated at admission—prior to current ICH—and at discharge from the hospital.

The continuous variables are presented as means and standard errors or medians and interquartile ranges. Categorical variables are presented as percentages.

Statistical tests such as the independent t test, Wilcoxon-Mann–Whitney, and chi-square were performed to account for differences between the pre-AI and post-AI cohorts.

We used a multiple logistic regression model to analyze the dichotomous variables while controlling for confounding covariates. Odds ratios are presented to indicate the likelihood of these clinical measures occurring between the time frames.

A two-sided p value of less than 0.05 was used to indicate statistical significance. Results are reported with 95% confidence intervals (CI).

All analyses were performed with R software, version 3.6.3.

Table 1 shows the complete comparison between the pre-AI group and the post-AI group for ICH, ischemic stroke (IS), and myocardial infarction (MI) demographic, comorbidity, medications, and clinical metrics—emergency severity index, length of stay, mortality at 30 and 12 days, and morbidity represented using Modified Ranking Scale.

No significant difference was observed for age, gender, smoking, diabetes mellitus, hypertension, chronic heart failure, or antihypertensive agent use for the ICH group (p = 0.062; p = 0.32, p = 1, p = 0.45, p = 0.37, p = 0.76, p = 0.36, respectively) or in any of the control groups of ischemic stroke and myocardial infarction (excluding p = 0.02 the male gender comparison between the pre-AI group and the post-AI group).

A significant difference was observed between the anticoagulants and anti-aggregant agents use in the pre-AI group compared to the post-AI group (41.8% vs 28.8%; p = 0.001) for the ICH dataset, which was constant for ischemic stroke and myocardial infarction datasets as well (p =  < 0.001 for both) (Table 3 in Supplements).

A significant difference was observed between the hemoglobin levels and platelet counts between the pre-AI group and post-AI group for the ICH dataset (12.9 vs 13.3; p = 0.007 and 216 vs 230; p = 0.009, respectively).

A significant decrease in the 30-day mortality rate was observed in the post-AI group compared to the pre-AI group (pre-AI 27.7% vs post-AI 17.5%, odds ratio = 0.48, CI of odd 0.29 to 0.79, p = 0.004), and a significant decrease in the 120-day mortality in the post-AI group in comparison to the pre-AI group (pre-AI 31.8% vs. post-AI 21.7%, odds ratio 0.58, CI of odds 0.37 to 0.91, p = 0.017) was observed for the ICH dataset.

No similar difference was observed for the ischemic stroke dataset (30 days odds ratio 1.4 [CI 0.73–2.85] p = 0.33; 120-day odds ratio 1.19 [CI 0.73–1.98] p = 0.5) or myocardial infarction dataset (30 days odds ratio 1.31 [CI 0.54–3.16] p = 0.55; 120-day odds ratio 1.72 [CI 0.79–3.81] p = 0.17) after controlling for significant confounders. The forest plot for mortality OR is shown in Fig. 2.

Sub-analysis of the participants with and without anticoagulation and anti-aggregation treatment was performed between the pre-AI group and post-AI group and its effect on 30- and 120-day mortality (Table 2).

Participants with mortality above 30 days post-admission were removed from the analysis for the 30-day mortality (pre-AI 228/289 participants’ analysis; post-AI 275/298 participants’ analysis).

Participants with mortality above 120 days post-admission were removed from the analysis for the 120-day mortality (pre-AI 252/289 participants’ analysis; post-AI 290/298 participants’ analysis).

Both groups showed a decrease in mortality percentage between pre-AI and post-AI, for both the 30- and 120-day mortality, with a significant decrease for the subgroup under anticoagulation and anti-aggregation treatment (− 52.3% p = 0.0114 for the 30-day mortality and − 43.9% p = 0.0156 for the 120-day mortality).

There was no significant difference between the baseline score at admission in the pre-AI group and post-AI group (0.77 vs 0.77, p = 0.94). Modified Rankin Scale at discharge showed a significant difference between the time periods with a decrease in the post-AI group (3.17 vs 2.84, p = 0.044) (Fig. 3). Modified Rankin Scale difference between discharge and admission, for each group separately, showed a significant difference between the pre-AI group and post-AI group (2.37 vs 2.03; p = 0.041).

The main limitation of this study is the fact this is a single-center, retrospective study focusing exclusively on ICH detection as well as our findings being correlative observations that may suggest, but does not prove, a causal relationship between AI implementation and decrease in mortality and morbidity decrease in patients with ICH. Future prospective studies such as a randomized clinical trial are necessary for further validation of these findings.

In conclusion, we have shown a correlative observation for the reduction in overall mortality and morbidity using a commercial, workflow-integrated computer-aided triage and prioritization AI solution on participants with intracranial hemorrhage. By flagging a life-threatening, time-sensitive pathology, the AI solution may improve overall reader efficiency, thus improving throughput, contributing to the timeliness in which radiologists can get to read scans with time-sensitive pathologies and possibly have the potential to enhance radiologists’ accuracy and subsequent clinical management.

The study was performed in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki—the Helsinki Department of Sheba Medical Center IRB, approved September 2020 # 3532–16-SMC.

Consent to participate

Not applicable for the consent section for all other parts.

Not applicable for the consent section for all other parts.

The authors declare that they have no competing interests.

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.