arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.02879cs.AI
收藏

MedCalc-Pro:使用大语言模型智能体解决复杂医学计算问题

MedCalc-Pro: Solving Complex Medical Calculations with LLM Agents

East China University of Science and Technology(华东理工大学)

Siran Zhao, Ruihui Hou, Ziyue Huai, Chennuo Zhang, Tong Ruan

AI总结:

研究针对医学计算评估大语言模型的基准过于简化的问题,提出新基准MedCalc-Pro及更通用智能体框架,含多场景任务设置和真实病例,新框架性能最佳,为评估应用LLMs提供新基准和方法。

摘要:

当前评估大语言模型在医学计算方面的基准大多基于简化设置。实际临床场景需求更多,为此提出MedCalc-Pro基准,涵盖三种任务设置,含2268个真实临床病例。还提出更通用智能体框架,跨模型比较结果显示该框架在三种任务设置中性能最佳,为医学计算评估应用LLMs提供新基准和方法。

英文摘要:

Current benchmarks for evaluating large language models (LLMs) in medical calculation are largely based on simplified settings, where each patient case corresponds to a single calculator and the required tool is explicitly specified in the query. However, real clinical scenarios often require multiple calculators for joint evaluation, nested-scale calculation, and fuzzy queries that do not directly specify the target calculator. To this end, we propose a new medical calculation benchmark, MedCalc-Pro, which covers three progressively challenging task settings: single-calculator, multi-calculator, and nested-calculator calculation settings. MedCalc-Pro contains 2,268 real-world clinical cases, covering 77 medical calculators across 14 clinical departments. Meanwhile, to address the limited performance of existing frameworks and methods in complex clinical scenarios, we further propose a more generalizable agent framework that supports multi-tool selection and nested-tool calling, while suppressing parameter error propagation through structured validation and evidence review. We conduct systematic comparisons across open-source, closed-source, and medical-specialized LLMs, and the results show that our framework achieves the best performance across all three task settings. This work provides a new benchmark and method for evaluating and applying LLMs in challenging medical calculation scenarios.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑