arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2605.10286cs.AI
收藏

AgentRx: 一种针对多模态临床预测任务的LLM代理基准研究

AgentRx: A Benchmark Study of LLM Agents for Multimodal Clinical Prediction Tasks

New York University Abu Dhabi(纽约大学阿布扎克分校)

Baraa Al Jorf, Farah E. Shamout

AI总结:

本文研究了LLM代理在多模态临床预测任务中的表现,发现单代理框架在处理多模态数据和校准方面优于多代理系统,强调了改进多代理协作的重要性。

摘要:

构建有效的临床决策支持系统需要合成复杂的异构多模态数据。此类模态包括时间电子健康记录数据、医学图像、放射报告和临床笔记。基于大语言模型(LLM)的代理在各种医疗任务中表现优异,尤其是在涉及文本模态的任务中。考虑到医疗数据在医院系统间的碎片化,协作代理框架提供了一种缓解数据共享挑战的有前途的方向。然而,LLM代理在多模态临床风险预测中的有效性尚未得到充分研究。在本工作中,我们使用大规模真实世界数据对LLM代理在临床预测任务中的表现进行了系统评估。我们评估了单模态和多模态设置下的性能,并量化了单代理系统与多代理系统之间的性能差距。我们的发现表明,单代理框架在处理多模态数据和校准方面优于朴素的多代理系统。这突显了改进多代理协作以更好地处理异构输入的重要性。通过开源我们的代码和评估框架,本工作提供了一个新的基准,以支持未来医疗领域代理系统的进一步发展。

英文摘要:

Building effective clinical decision support systems requires the synthesis of complex heterogeneous multimodal data. Such modalities include temporal electronic health records data, medical images, radiology reports, and clinical notes. Large language model (LLM)-based agents have shown impressive performance in various healthcare tasks, especially those involving textual modalities. Considering the fragmentation of healthcare data across hospital systems, collaborative agent frameworks present a promising direction to mitigate data sharing challenges. However, the effectiveness of LLM agents for multimodal clinical risk prediction remains largely unexamined. In this work, we conduct a systematic evaluation of LLM-based agents for clinical prediction tasks using large-scale real-world data. We assess performance in unimodal and multimodal settings and quantify performance gaps between single agent and multi-agent systems. Our findings highlight that single agent frameworks outperform naive multi-agent systems, are better at handling multimodal data, and are better calibrated. This underscores a critical need for improving multi-agent collaboration to better handle heterogeneous inputs. By open-sourcing our code and evaluation framework, this work offers a new benchmark to support future developments relating to agentic systems in healthcare.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑