American Journal of Emergency Medicine 94 (2025) 46–49

Contents lists available at ScienceDirect

American Journal of Emergency Medicine
journal homepage: www.elsevier.com/locate/ajem

Performance of the artiﬁcial intelligence-based Swiss medical
assessment system versus Manchester triage system in the
emergency department: A retrospective analysis
Gregor Lindner a,b,⁎, Svenja Ravioli a
a
b

Department of Emergency Medicine, Kepler Universitätsklinikum GmbH, Johannes Kepler University, Linz, Austria
Department of Emergency Medicine, Inselspital, University of Bern, Bern, Switzerland

a r t i c l e

i n f o

Article history:
Received 5 March 2025
Received in revised form 7 April 2025
Accepted 10 April 2025
Available online xxxx
Keywords:
Artiﬁcial intelligence
Chatbot
Emergency
Triage

a b s t r a c t
Background: The emergence of artiﬁcial intelligence (AI) offers new opportunities for applications in emergency
medicine, including patient triage. This study evaluates the performance of the Swiss Medical Assessment System
(SMASS), an AI-based decision-support tool for rapid patient assessment, in comparison with the wellestablished Manchester Triage System (MTS).
Methods: In this retrospective analysis, patients aged 18 years or above presenting to the Department of
Emergency Medicine at Kepler University Hospital in Linz, Austria, during November and December 2024 with
non-traumatic complaints were included. Each patient underwent emergency triage using MTS, conducted by
a registered nurse, with SMASS applied in parallel. SMASS had no inﬂuence on clinical decision-making.
Results: In the study period, 1021 patients were triaged with both MTS and SMASS. The mean patient age was
60 years (SD: 21), and 53 % were women. Of the patients categorized as “orange” by MTS, 19 % were classiﬁed
as non-urgent by SMASS. Conversely, 28 % of the patients triaged as “green” by MTS were classiﬁed as urgent
by SMASS. Additionally, 23 % of patients classiﬁed as non-urgent by SMASS required hospitalization following
emergency department evaluation and treatment. Agreement between SMASS and MTS in triaging emergency
patients was low as measured by a Cohen's kappa of 0.167.
Conclusions: In this study of patients presenting to a large tertiary-care emergency department, SMASS demonstrated considerable discrepancies in triage classiﬁcation compared to MTS, with signiﬁcant rates of both
over- and undertriage. Further validation is necessary before integrating AI-based triage tools into routine clinical
practice.
© 2025 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY license (http://
creativecommons.org/licenses/by/4.0/).

1. Introduction
Triage is a well-established process in emergency medicine, designed to prioritize patients for treatment based on the acuity and severity of their presenting symptoms or conditions. It is a fundamental
process in emergency care, as patient demand often exceeds the available resources, in particular the supply of nurses and physicians [1]. Various triage systems were proposed, many of which demonstrated
reasonable validity [2]. Emergency Severity Index (ESI), the Canadian
Triage and Acuity Scale (CTAS), and the Manchester Triage System
(MTS) are among the most widely used international triage tools
[3-5]. These triage systems are algorithm-based, typically relying on

⁎ Corresponding authors at: Dep. of Emergency Medicine Kepler Universitätsklinikum
GmbH, Krankenhausstrasse 9, 4020 Linz, Austria.
E-mail address: lindner.gregor@gmail.com (G. Lindner).

presenting symptoms and vital signs to categorize patient acuity. A recent systematic review found a similar performance for the MTS, ESI,
and CTAS [6]. While these systems exhibited high sensitivity in identifying patients at risk of emergency department mortality, they were less
effective in predicting the need for intensive care or in-hospital mortality [6].
New technologies, particularly artiﬁcial intelligence (AI), hold significant potential for enhancing current triage standards in emergency
medicine [7]. AI-assisted systems are able to integrate a much larger
volume of data to optimize triage sensitivity. Initial applications of AI
in emergency triage have shown promising results, particularly in the
assessment of neurological patients [8]. A recent study comparing traditional nurse-based triage to a large language model (LLM)-based generative AI model found no signiﬁcant difference in triage accuracy, but
AI-based triage demonstrated lower rates of undertriage [9]. However,
there remains a strong need for further research comparing innovative
AI-driven triage systems to established traditional methods.

https://doi.org/10.1016/j.ajem.2025.04.023
0735-6757/© 2025 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).

G. Lindner and S. Ravioli

American Journal of Emergency Medicine 94 (2025) 46–49

Of the 196 patients categorized as “orange” by MTS, 37 (19 %) were
classiﬁed as “non-urgent” by SMASS. Conversely, of the 341 patients
triaged as “green” by MTS, 95 (28 %) were deemed “urgent” by
SMASS. Additionally, two (13 %) of the 15 patients categorized as
“blue” by MTS were classiﬁed as “urgent” by SMASS.
In terms of agreement between triage of emergency patients using
SMASS versus MTS Cohen's kappa was 0.167 (SE 0.021, 95 % CI:
0.125–0.208). In differentiating cases as being “acute” or “non-acute”
SMASS showed a sensitivity of 62 % and a speciﬁcity of 73 % with a positive predictive value of 81 % and a negative predictive value of 50 %.
Among the 515 patients identiﬁed as non-urgent by SMASS, 119
(23 %) required hospitalization after emergency department evaluation
and treatment.
A Sankey diagram illustrating the results of MTS and SMASS triage
for all 1021 patients is provided in Fig. 1.

This study aimed to compare patient triage outcomes at a large tertiary care emergency department using the Swiss Medical Assessment
System (SMASS) alongside the conventional MTS.
2. Methods
SMASS is a software tool designed to assist healthcare professionals
(SMASS Professional) and laypersons (SMASS Pathﬁnder) in rapidly
assessing medical conditions and recommending appropriate treatment
pathways (https://in4medicine.ch/ﬁles/Bilder/Allgemein/SMASS_Info.
pdf). It comprises an editor, knowledge databases, and a runtime environment. The software is cloud-based and certiﬁed as a medical device
under European Union standards. Further details on SMASS can be
found on the developer's website: https://in4medicine.ch/smass-107.
html.
In this retrospective analysis, patients aged 18 years or above
presenting to the Department of Emergency Medicine at Kepler
University Hospital in Linz, Austria, were eligible. Patients with
suspected stroke, trauma-related complaints and patients accompanied by pre-hospital emergency medicine physicians were excluded. SMASS was evaluated over a limited period in November
and December 2024 to assess its usability among healthcare professionals and potential clinical applications. All patients underwent
initial emergency triage by a registered nurse trained in MTS. Patients who simultaneously underwent triage with SMASS were included in this study.
Data was collected in a pseudonymized format. Collected data included age, sex, reason for emergency department admission, MTS
and SMASS triage results, length of stay, and in-hospital mortality.
Data are presented as mean and standard deviation or median and
interquartile range, as appropriate. In order to assess agreement between triage by SMASS compared to MTS Cohen's kappa was calculated.
Sensitivity, speciﬁcity, positive and negative predictive values were calculated deﬁning MTS red, orange and yellow as “acute” and MTS green
and blue as “non-acute” while SMASS categories “immediate medical
treatment” and “emergency” as “acute” and “medical treatment
today” and “medical treatment non-urgent” as “non-acute”. All statistics
were calculated using JASP, version 0.19.3. The study was approved by
the local ethics committee, the “Ethikkommission der Johannes Kepler
Universität,” and the requirement for individual patient consent was
waived due to the retrospective design of the study (protocol number
1385/2024).

4. Discussion
This study including 1021 emergency department patients admitted
at a large university hospital, found that SMASS triage resulted in significant rates of both over- and undertriage compared to the wellestablished MTS.
Several triage systems have been developed for emergency patients,
each incorporating different variables such as vital signs, expected resource utilization, or presenting symptoms. While established triage
systems perform similarly in terms of sensitivity [2,6], mistriage remains a concern for both adult and pediatric populations [3,10-12]. Various approaches have been explored to minimize mistriage and
enhance patient safety [13,14]. The integration of AI into emergency triage appears promising, with initial pilot studies showing encouraging
results [9,15,16].
While AI-assisted triage may enable a faster and more reliable triage
process, this study underlines the necessity of rigorous real-world testing for usability, reproducibility, and patient safety. Notably, 19 % of patients classiﬁed as “orange” by MTS - requiring physician evaluation
within 10 min - were categorized as “non-urgent” by SMASS, posing a
potential safety risk. Additionally, 23 % of patients deemed “non-urgent”
by SMASS required hospitalization, suggesting a signiﬁcant discrepancy
in triage result. Conversely, nearly one-third of patients categorized as
“green” or “blue” by MTS were classiﬁed as urgent by SMASS. This indicates that SMASS failed on both ends of the spectrum: On the one hand
patients with rather minor conditions were apparently over-triaged
while others with potentially dangerous presentations were undertriaged. Moreover, agreement between the two triage systems was
low as indicated by Cohen's kappa.
While overall performance of SMASS appeared promising, the tool
apparently needs training in the acute care setting based on real patient
data from emergency department patients in order to decrease the relevantly high numbers of relevant over- and undertriage of patients seen
in the current study. This is crucial if SMASS should be used for triage of
emergency medicine patients. In general. It should be part of training of
every new triage algorithm being AI-based or not to test and train the
model on a large number of real emergency medicine cases including
the full variety of the ﬁeld ranging from rather minor cases to lifethreatening conditions. This is the case in every day emergency medicine and thus needs to be considered, In order to achieve this emergency
physicians and nursing staff should be cooperated with in the development of these models in order to achieve optimal results.
One possible explanation of the meager results of SMASS compared
to the “gold-standard” MTS might be that the inclusion of vital signs is
missing in the SMASS model. This could potentially help in differentiating patients. However, whether the “mistriage” of patients by SMASS
compared to MTS is systematic or random cannot be answered by the
current data. In addition, further research will be needed in order to assess whether SMASS is prone to over- or undertriage in certain patient
groups.

3. Results
Between November 28 and December 8, 2024, 1021 patients presenting to the Department of Emergency Medicine at Kepler University
Hospital were triaged with both MTS and SMASS. Of all patients triaged
with both systems, 540 (53 %) were women. The mean patient age was
60 years (SD: 21).
The most common presenting symptom/diagnosis categories were
as follows: cardiology (158; 15 %), musculoskeletal (123; 12 %), neurology (122; 12 %), respiratory (122; 12 %), dermatology (102; 10 %), general and visceral surgery (96; 9 %), gastroenterology (81; 8 %),
nonspeciﬁc (80; 8 %), and urology/nephrology (74, 7 %).
A total of 731 (72 %) patients were treated as outpatients, while 290
(28 %) were admitted as inpatients. The median length of stay for inpatients was four days (IQR: 2–8). In-hospital mortality occurred in 15
(1 %) patients.
The MTS triage distribution was as follows: two patients (0.2 %)
were classiﬁed as “red”, 196 (19 %) as “orange”, 467 (46 %) as “yellow”,
341 (33 %) as “green” and 15 (1 %) as “blue”.
The SMASS triage distribution was as follows: 385 (38 %) patients
were categorized as requiring “immediate medical treatment”, 121
(12 %) as “emergency”, 331 (32 %) as requiring “medical treatment
today” and 184 (18 %) as “medical treatment non-urgent”.
47

G. Lindner and S. Ravioli

American Journal of Emergency Medicine 94 (2025) 46–49

Fig. 1. Results of MTS and SMASS triage for 1′021 emergency medicine patients. The diagram shows relevant over- and under-triage of patients by SMASS compared to the gold-standard of
MTS. Detailed numbers are given in the text. (For interpretation of the references to colour in this ﬁgure legend, the reader is referred to the web version of this article.)

Although well-established triage systems such as MTS, ESI, and CTAS
have recognized limitations, they remain the gold standard for emergency triage, offering high reliability for identifying patients at risk of
rapid deterioration. New triage systems need to be benchmarked
against these established methods to ensure accuracy and patient
safety.
The present study has several limitations: Firstly, due to its retrospective design data quality is somehow limited. Furthermore, the
study does not aim to investigate the functioning of the model, algorithms and decision logic of SMASS.

References
[1] Lindner G, Woitok BK. Emergency department overcrowding : analysis and strategies to manage an international phenomenon. Wien Klin Wochenschr. 2021;133:
229–33.
[2] Zachariasse JM, van der Hagen V, Seiger N, Mackway-Jones K, van Veen M, Moll HA.
Performance of triage systems in emergency care: a systematic review and metaanalysis. BMJ Open. 2019;9:e026471.
[3] Sax DR, Warton EM, Mark DG, Vinson DR, Kene MV, Ballard DW, et al. Evaluation of
the emergency severity index in US emergency departments for the rate of
Mistriage. JAMA Netw Open. 2023;6:e233404.
[4] Alnujaidi M, Aldibane R, Alosaimi R, Kashmeeri R, Altalhi E, Shalabi A. 71. Factors
predicting hospital admission for non-urgent patients triaged with the Canadian triage and acuity scale (CTAS) in the emergency department. A retrospective study in
tertiary Center in Makkah, KSA. Eur J Emerg Med. 2020;27:e16–7.
[5] Zaboli A, Ausserhofer D, Pfeifer N, Solazzo P, Magnarelli G, Siller M, et al. Triage of
patients with fever: the Manchester triage system’s predictive validity for sepsis or
septic shock and seven-day mortality. J Crit Care. 2020;59:63–9.
[6] Hinson JS, Martinez DA, Cabral S, George K, Whalen M, Hansoti B, et al. Triage performance in emergency medicine: a systematic review. Ann Emerg Med. 2019;74:
140–52.
[7] Siira E, Johansson H, Nygren J. Mapping and summarizing the research on AI Systems for Automating Medical History Taking and Triage: scoping review. J Med Internet Res. 2025;27:e53741.
[8] Alessandro L., Crema S., Castiglione J.I., Dossi D., Eberbach F., Kohler A., et al. Validation of an artiﬁcial intelligence-powered virtual assistant for emergency triage in
neurology. Neurologist 2025. Online ahead of print.
[9] Arslan B, Nuhoglu C, Satici MO, Altinbilek E. Evaluating LLM-based generative AI
tools in emergency triage: a comparative study of ChatGPT plus, copilot pro, and triage nurses. Am J Emerg Med. 2024;89:174–81.
[10] Mirhaghi A, Mazlom R, Heydari A, Ebrahimi M. The reliability of the Manchester triage system (MTS): a meta-analysis. J Evid Based Med. 2017;10:129–35.
[11] Frankenberger WD, Zorc JJ, Ten Have ED, Brodecki D, Faig WG. Triage accuracy in pediatrics using the emergency severity index. J Emerg Nurs. 2024;50:207–14.
[12] Geanacopoulos AT, Peltz A, Melton K, Neuman MI, Gutman CK, Walsh KE, et al. Pediatric triage accuracy in pediatric and general emergency departments. Hosp
Pediatr. 2025;15:37–45.
[13] Levkowitz E, Gibson R, Xu H, Zhang LF, Eskine K, Buck B, et al. Optimizing triage:
assessing shock index, pediatric age-adjusted as an adjunct to improve emergency
severity index Mistriage. Pediatr Emerg Care. 2025;41:11–6.
[14] Wretborn J, Munir-Ehrlington S, Horlin E, Wilhelms DB. Addition of the clinical
frailty scale to triage tools and early warning scores improves mortality prognostication at 30 days: a prospective observational multicenter study. J Am Coll Emerg Phys
Open. 2024;5:e13244.

5. Conclusion
This study compared MTS and SMASS triage outcomes in emergency
patients and found substantial discrepancies, with a high rate of overand undertriage using SMASS. These ﬁndings highlight the importance
of thorough validation before implementing new triage technologies
to ensure patient safety.
Declaration of generative AI in scientiﬁc writing
None.
CRediT authorship contribution statement
Gregor Lindner: Writing – review & editing, Writing – original draft,
Supervision, Methodology, Investigation, Formal analysis, Data curation,
Conceptualization. Svenja Ravioli: Writing – review & editing, Writing
– original draft, Methodology, Investigation, Conceptualization.
Declaration of competing interest
GL and SR declare no conﬂict of interest in relation to this
submission.
48

G. Lindner and S. Ravioli

American Journal of Emergency Medicine 94 (2025) 46–49

[15] Haim G.B., Saban M., Barash Y., Cirulnik D., Shaham A., Eisenman B.Z., et al. Evaluating large language model-assisted emergency triage: a comparison of acuity assessments by GPT-4 and medical experts. J Clin Nurs 2024. Nov 28. doi: . https://doi.
org/10.1111/jocn.17490. Online ahead of print.

[16] Williams CYK, Zack T, Miao BY, Sushil M, Wang M, Kornblith AE, et al. Use of a large
language model to assess clinical acuity of adults in the emergency department.
JAMA Netw Open. 2024;7:e248895.

49