arXiv 2504.024042025-04cs.CL

AnesSuite: 一个用于LLM麻醉推理的全面基准和数据集套件

AnesSuite: A Comprehensive Benchmark and Dataset Suite for Anesthesiology Reasoning in LLMs

School of Computer Science, National Engineering Research Center for Multimedia Software and Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University, China（计算机学院、多媒体软件国家工程研究中心和多媒体与网络通信工程湖北省重点实验室、武汉大学）；Department of Anesthesiology, Zhejiang Cancer Hospital, China（麻醉科、浙江省癌症医院）；Institute of Medicine, Chinese Academy of Sciences, Hangzhou, Zhejiang, China（医学研究所、中国科学院、杭州、浙江）；Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore（李光耀医学院、南洋理工大学、新加坡）；Centre of AI in Medicine, Nanyang Technological University, Singapore（医学人工智能中心、南洋理工大学、新加坡）；Department of Anesthesiology, First People’s Hospital of Yunnan Province, China（麻醉科、云南第一人民医院）；Kunming University of Science and Technology, China（昆明理工大学）

Xiang Feng, Wentao Jiang, Zengmao Wang, Yong Luo, Pingbo Xu, Baosheng Yu, Hua Jin, Jing Zhang

备注：: Accepted in ICLR 2026; 47 pages, 12 figures, 26 tables;

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

AnesSuite为LLM麻醉推理提供首个全面基准和数据集，开发Morpheus模型在麻醉领域实现显著性能提升。

摘要：

大语言模型（LLMs）在医疗领域的应用已经引起了广泛关注，然而它们在更专业的领域如麻醉学中的推理能力仍缺乏深入研究。为填补这一空白，我们引入了AnesSuite，这是首个专门为LLM中的麻醉推理设计的全面数据集套件。该套件包含AnesBench，一个专门评估麻醉相关推理的评估基准，涵盖三个层次：事实检索（System 1）、混合推理（System 1.x）和复杂决策制定（System 2）。除了这个基准，该套件还包括三个训练数据集，为持续预训练（CPT）、监督微调（SFT）和带有可验证奖励的强化学习（RLVR）提供基础设施。利用该套件，我们开发了Morpheus，首个用于麻醉推理的基线模型集合。尽管经过有限的SFT和群体相对策略优化（GRPO）训练，Morpheus不仅在麻醉领域实现了与大规模模型相当的显著改进，还展示了在通用医疗和广泛领域基准上的增强推理能力。此外，通过全面的评估和实验，我们分析了影响麻醉推理性能的关键因素，包括模型特性、训练策略和训练数据。AnesSuite和Morpheus将在https://github.com/MiliLab/AnesSuite上开源。

英文摘要：

The application of large language models (LLMs) in the medical field has garnered significant attention, yet their reasoning capabilities in more specialized domains like anesthesiology remain underexplored. To bridge this gap, we introduce AnesSuite, the first comprehensive dataset suite specifically designed for anesthesiology reasoning in LLMs. The suite features AnesBench, an evaluation benchmark tailored to assess anesthesiology-related reasoning across three levels: factual retrieval (System 1), hybrid reasoning (System 1.x), and complex decision-making (System 2). Alongside this benchmark, the suite includes three training datasets that provide an infrastructure for continued pre-training (CPT), supervised fine-tuning (SFT), and reinforcement learning with verifiable rewards (RLVR). Leveraging this suite, we develop Morpheus, the first baseline model collection for anesthesiology reasoning. Despite undergoing limited training with SFT and group relative policy optimization (GRPO), Morpheus not only achieves substantial improvements in anesthesiology that rival larger-scale models, but also demonstrates enhanced reasoning capabilities across general medical and broad-domain benchmarks. Furthermore, through comprehensive evaluations and experiments, we analyze the key factors influencing anesthesiology reasoning performance, including model characteristics, training strategies and training data. Both AnesSuite and Morpheus will be open-sourced at https://github.com/MiliLab/AnesSuite.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。