arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2509.23379cs.CLcs.AIcs.CV

CCD:通过临床对比解码缓解放射学多模态大语言模型中的幻觉

CCD: Mitigating Hallucinations in Radiology MLLMs via Clinical Contrastive Decoding

School of Computing Science, University of Glasgow(计算科学学院,格拉斯哥大学)

Xi Zhang, Zaiqiao Meng, Jake Lever, Edmond S. L. Ho

AI总结:

针对放射学多模态大语言模型的医学幻觉问题,提出无需训练和检索的临床对比解码(CCD)框架,通过双阶段对比机制整合专家模型信号,在MIMIC-CXR上使RRG模型RadGraph-F1提升高达17%。

摘要:

多模态大语言模型(MLLMs)通过整合视觉感知与自然语言理解,近期在放射学领域取得了显著进展。然而,它们常常生成临床上无依据的描述,即所谓的医学幻觉,这在需要准确性和基于图像输出的医疗应用中构成严重风险。通过实证分析,我们发现提示引发的幻觉在放射学多模态大语言模型中仍然普遍存在,主要归因于对临床部分的过度敏感。为解决这一问题,我们引入了临床对比解码(CCD),这是一种无需训练且无需检索的推理框架,整合了来自任务特定放射学专家模型的结构化临床信号。CCD引入了一种双阶段对比机制,在生成过程中细化令牌级对数概率,从而在不修改基础多模态大语言模型的情况下增强临床保真度。在三个数据集和多个模型上的实验表明,CCD在放射学报告生成(RRG)任务上持续提升了整体性能。在MIMIC-CXR数据集上,当应用于最先进的RRG模型时,它在RadGraph-F1指标上实现了高达17%的提升。我们的方法为缓解医学幻觉提供了一种轻量级且可泛化的解决方案,有效地在放射学中架起了专家模型与多模态大语言模型之间的桥梁。

英文摘要:

Multimodal large language models (MLLMs) have recently achieved remarkable progress in radiology by integrating visual perception with natural language understanding. However, they often generate clinically unsupported descriptions, known as medical hallucinations, which pose serious risks in medical applications that demand accuracy and image-grounded outputs. Through empirical analysis, we find that prompt-induced hallucinations remain prevalent in radiology MLLMs, largely due to over-sensitivity to clinical sections. To address this, we introduce Clinical Contrastive Decoding (CCD), a training-free and retrieval-free inference framework that integrates structured clinical signals from task-specific radiology expert models. CCD introduces a dual-stage contrastive mechanism to refine token-level logits during generation, thereby enhancing clinical fidelity without modifying the base MLLM. Experiments on three datasets and multiple models demonstrate that CCD consistently improves overall performance on radiology report generation (RRG). On the MIMIC-CXR dataset, it yields up to a 17% improvement in RadGraph-F1 when applied to state-of-the-art RRG models. Our approach provides a lightweight and generalisable solution for mitigating medical hallucinations, effectively bridging expert models and MLLMs in radiology.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑