arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.13610cs.AIcs.CL
收藏

MeNTi:通过嵌套工具调用桥接医学计算器与LLM智能体

MeNTi: Bridging Medical Calculator and LLM Agent with Nested Tool Calling

Shanghai Jiao Tong University(上海交通大学);Peking University(北京大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);SenseTime Research(商汤科技研究院)

Yakun Zhu, Shaohang Wei, Xu Wang, Kui Xue, Xiaofan Zhang, Shaoting Zhang

AI总结:

提出MeNTi通用智能体架构,通过元工具和嵌套调用机制增强LLM在医学计算器任务中的工具利用,并构建CalcQA基准验证其显著性能提升。

摘要:

将工具集成到大型语言模型(LLMs)中促进了其广泛应用。尽管如此,在专业下游任务场景中,仅依赖工具不足以完全应对现实世界的复杂性。这尤其限制了LLMs在医学等领域的有效部署。本文聚焦于医学计算器的下游任务,这些计算器使用标准化测试来评估个体健康状况。我们提出了MeNTi,一种通用的LLM智能体架构。MeNTi集成了专门的医学工具包,并采用元工具和嵌套调用机制来增强LLM的工具利用能力。具体而言,它实现了灵活的工具选择和嵌套工具调用,以解决复杂医学场景中面临的实际问题,包括计算器选择、槽位填充和单位转换。为了评估LLMs在计算器场景的临床过程中进行定量评估的能力,我们引入了CalcQA基准。该基准要求LLMs使用医学计算器进行计算并评估患者健康状况。CalcQA由专业医生构建,包含100对病例-计算器组合,并辅以包含281个医学工具的工具包。实验结果表明,我们的框架带来了显著的性能提升。这项研究为LLMs在医学高要求场景中的应用开辟了新方向。

英文摘要:

Integrating tools into Large Language Models (LLMs) has facilitated the widespread application. Despite this, in specialized downstream task contexts, reliance solely on tools is insufficient to fully address the complexities of the real world. This particularly restricts the effective deployment of LLMs in fields such as medicine. In this paper, we focus on the downstream tasks of medical calculators, which use standardized tests to assess an individual's health status. We introduce MeNTi, a universal agent architecture for LLMs. MeNTi integrates a specialized medical toolkit and employs meta-tool and nested calling mechanisms to enhance LLM tool utilization. Specifically, it achieves flexible tool selection and nested tool calling to address practical issues faced in intricate medical scenarios, including calculator selection, slot filling, and unit conversion. To assess the capabilities of LLMs for quantitative assessment throughout the clinical process of calculator scenarios, we introduce CalcQA. This benchmark requires LLMs to use medical calculators to perform calculations and assess patient health status. CalcQA is constructed by professional physicians and includes 100 case-calculator pairs, complemented by a toolkit of 281 medical tools. The experimental results demonstrate significant performance improvements with our framework. This research paves new directions for applying LLMs in demanding scenarios of medicine.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑