arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2504.01201cs.CLcs.AIcs.HC
收藏

医学大型语言模型极易受到干扰

Medical large language models are easily distracted

NYU Langone Medical Center(纽约大学朗格尼医学中心);The University of Texas at Austin(德克萨斯大学奥斯汀分校);Washington University School of Medicine in St. Louis(圣路易斯华盛顿大学医学院);New York University(纽约大学)

Krithik Vishwanath, Anton Alyakin, Daniel Alexander Alber, Jin Vivian Lee, Douglas Kondziolka, Eric Karl Oermann

AI总结:

本文提出MedDistractQA基准评估LLM在医学场景中抗干扰能力,发现无关信息可使准确率降低17.9%,且RAG和微调无法解决此问题,揭示了LLM缺乏区分相关与无关临床信息的逻辑机制。

摘要:

大型语言模型(LLM)有潜力改变医学,但现实世界的临床场景包含可能阻碍性能的无关信息。辅助技术(如环境听写)的兴起,能够从现场患者就诊中自动生成草稿笔记,可能会引入额外噪声,因此评估LLM过滤相关数据的能力变得至关重要。为此,我们开发了MedDistractQA,这是一个使用嵌入了模拟现实世界干扰的USMLE风格问题的基准。我们的发现表明,干扰性陈述(在非临床语境中使用具有临床含义的多义词,或提及不相关的健康状况)可使LLM的准确率降低高达17.9%。常见的旨在提高模型性能的解决方案,如检索增强生成(RAG)和医学微调,并未改变这一影响,在某些情况下还引入了自身的混杂因素并进一步降低了性能。我们的发现表明,LLM本质上缺乏区分相关与无关临床信息所需的逻辑机制,这给现实世界的应用带来了挑战。MedDistractQA及我们的结果凸显了需要稳健的缓解策略来增强LLM对无关信息的抵御能力。

英文摘要:

Large language models (LLMs) have the potential to transform medicine, but real-world clinical scenarios contain extraneous information that can hinder performance. The rise of assistive technologies like ambient dictation, which automatically generates draft notes from live patient encounters, has the potential to introduce additional noise making it crucial to assess the ability of LLM's to filter relevant data. To investigate this, we developed MedDistractQA, a benchmark using USMLE-style questions embedded with simulated real-world distractions. Our findings show that distracting statements (polysemous words with clinical meanings used in a non-clinical context or references to unrelated health conditions) can reduce LLM accuracy by up to 17.9%. Commonly proposed solutions to improve model performance such as retrieval-augmented generation (RAG) and medical fine-tuning did not change this effect and in some cases introduced their own confounders and further degraded performance. Our findings suggest that LLMs natively lack the logical mechanisms necessary to distinguish relevant from irrelevant clinical information, posing challenges for real-world applications. MedDistractQA and our results highlights the need for robust mitigation strategies to enhance LLM resilience to extraneous information.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑