Small Language Models for Emergency Departments Decision Support: A Benchmark Study
Department of Electrical and Software Engineering, University of Calgary, Calgary, AB, Canada（电气与软件工程系，卡尔加里大学）；Department of Emergency Medicine, University of Calgary, Calgary, AB, Canada（急诊医学系，卡尔加里大学）；Rockview General Hospital, Calgary, AB, Canada（罗克维尔医院）
Zirui Wang, Jiajun Wu, Braden Teitge, Jessalyn Holodinsky, Steve Drew
备注：
Accepted to 2025 IEEE International Conference on Autonomous and Trusted Computing (ATC 2025)
Large language models (LLMs) have become increasingly popular in medical domains to assist physicians with a variety of clinical and operational tasks. Given the fast-paced and high-stakes environment of emergency departments (EDs), small language models (SLMs), characterized by a reduction in parameter count compared to LLMs, offer significant potential due to their inherent reasoning capability and efficient performance. This enables SLMs to support physicians by providing timely and accurate information synthesis, thereby improving clinical decision-making and workflow efficiency. In this paper, we present a comprehensive benchmark designed to identify SLMs suited for ED decision support, taking into account both specialized medical expertise and broad general problem-solving capabilities. In our evaluations, we focus on SLMs that have been trained on a mixture of general-domain and medical corpora. A key motivation for emphasizing SLMs is the practical hardware limitations, operational cost constraints, and privacy concerns in the typical real-world deployments. Our benchmark datasets include MedMCQA, MedQA-4Options, and PubMedQA, with the medical abstracts dataset emulating tasks aligned with real ED physicians' daily tasks. Experimental results reveal that general-domain SLMs surprisingly outperform their medically fine-tuned counterparts across these diverse benchmarks for ED. This indicates that for ED, specialized medical fine-tuning of the model may not be required.
ED-Copilot: Reduce Emergency Department Wait Time with Language Model Diagnostic Assistance
Language Technologies Institute, Carnegie Mellon University（卡内基梅隆大学语言技术研究所）；University of California, Berkeley（加州大学伯克利分校）；University of California, San Francisco（加州大学旧金山分校）；Department of EECS, University of California, Berkeley（加州大学伯克利分校电子工程与计算机科学系）
Validating Emergency Department Admission Predictions Based on Local Data Through MIMIC-IV
Hellenic Open University（希腊开放大学）；Sismanogleio General Hospital（西斯马诺格利奥综合医院）；National and Kapodistrian University of Athens（雅典国立卡波迪斯特里亚大学）；Amalia Fleming General Hospital（阿玛利亚·弗莱明综合医院）
Multimodal Clinical Benchmark for Emergency Care (MC-BEC): A Comprehensive Benchmark for Evaluating Foundation Models in Emergency Medicine
Harvard Medical School（哈佛医学院）；Stanford University（斯坦福大学）；Harvard John A. Paulson School of Engineering and Applied Sciences, Harvard University（哈佛大学约翰·A·保尔森工程与应用科学学院）
Development of a Large Language Model-based Multi-Agent Clinical Decision Support System for Korean Triage and Acuity Scale (KTAS)-Based Triage and Treatment Planning in Emergency Departments
John Bapst Memorial High School（约翰·巴普斯特纪念高中）；Minerva University（密涅瓦大学）
Why Do Self-Harm Prediction Models Struggle to Generalise? Lexical and Semantic Variations in Emergency Department Triage Notes
School of Computing and Information Systems, The University of Melbourne, Australia（墨尔本大学计算与信息系）；Orygen, The National Centre of Excellence in Youth Mental Health, Australia（青年心理健康国家卓越研究中心）；Centre for Youth Mental Health, The University of Melbourne, Australia（青年心理健康中心）；Centre for Digital Transformation of Health, The University of Melbourne, Australia（健康数字化转型中心）
Transferable Self-Harm Surveillance from Emergency Department Triage Notes Using an Evidence-Augmented Machine Learning Approach
School of Computing and Information Systems, University of Melbourne（墨尔本大学计算与信息系）；Orygen；Centre for Youth Mental Health, University of Melbourne（墨尔本大学青年心理健康中心）；Centre for Digital Transformation of Health, University of Melbourne（墨尔本大学医疗数字化转型中心）
Emergency Department Decision Support using Clinical Pseudo-notes
Department of Computational Medicine, University of California, Los Angeles, CA 90095 USA（计算医学系，加州大学洛杉矶分校）；Department of Neurosurgery, University of California, Los Angeles, CA 90095 USA（神经外科系，加州大学洛杉矶分校）；Department of Electrical and Computer Engineering, University of California at Los Angeles, Los Angeles, CA 90095 USA（电气与计算机工程系，加州大学洛杉矶分校）；Harbor-UCLA Medical Center, Department of Emergency Medicine, Torrance, CA（Harbor-UCLA医疗中心，急诊医学部）；University of California, Los Angeles, Department of Emergency Medicine, Los Angeles, California（加州大学洛杉矶分校，急诊医学部）；Department of Statistics and Data Science University of California, Los Angeles（统计与数据科学系，加州大学洛杉矶分校）；Department of Natural Sciences International Christian University, Mitaka, Tokyo, Japan（自然科学系，国际基督教大学）
Multi-Modal Perceiver Language Model for Outcome Prediction in Emergency Department
Qatar Computing Research Institute（卡塔尔计算研究所）；Hamad Bin Khalifa University（哈马德·本·哈利法大学）；LIMTIC Laboratory（LIMTIC实验室）；UTM University（突尼斯马纳大学）；Higher institute of computer science of Medenine（梅德宁高等计算机科学学院）；Gabes University（加贝斯大学）；Sidra Medicine（锡德拉医学中心）
Detection of sepsis during emergency department triage using machine learning
Mednition Inc.（麦迪逊公司）；Sutter Mills-Peninsula Medical Center（萨特米尔半岛医疗中心）；Ascension Health（阿森松医疗）；Adventist Health White Memorial（基督复临安息日会怀特纪念医疗中心）；San Mateo Medical Center（圣马特奥医疗中心）；El Camino Hospital（埃尔卡米诺医院）；Emergency Nurses Association（急诊护士协会）；Duke University（杜克大学）
ER-Reason: A Benchmark Dataset for LLM Clinical Reasoning in the Emergency Room
University of California, Berkeley（加州大学伯克利分校）；University of California, San Francisco（加州大学旧金山分校）；Duke University（杜克大学）；University of Alberta（阿尔伯塔大学）；Beth Israel Deaconess Medical Center（贝斯以色列德aconess医疗中心）；UC Berkeley and UCSF（伯克利大学和旧金山分校）
