arXiv 2512.019222025-12-01cs.CV

Med-VCD: 通过视觉对比解码缓解医疗大视觉语言模型的幻觉问题

Med-VCD: Mitigating Hallucination for Medical Large Vision Language Models through Visual Contrastive Decoding

Department of computer science, University of Central Florida, Orlando, USA（计算机科学系，中央佛罗里达大学）；Department of Bioengineering, University of Pennsylvania, Philadelphia, PA, USA（生物工程系，宾夕法尼亚大学）；Department of electrical engineering, Columbia university, New York, NY, USA（电气工程系，哥伦比亚大学）；Technical University of Applied Sciences Regensburg, Regensburg, Germany（应用科学技术大学（雷根斯堡））；Department of Surgery, University of Calgary, Calgary, Alberta, Canada（外科系，卡尔加里大学）；University College of Nabi Akram, Tabriz, Iran（纳比阿克兰大学）；School of Electrical Engineering, Iran University of Science and Technology, Tehran, Iran（电气工程学院，伊朗科学技术大学）

Zahra Mahdavi, Zahra Khodakaramimaghsoud, Hooman Khaloo, Sina Bakhshandeh Taleshani, Erfan Hashemi, Javad Mirzapour Kaleybar, Omid Nejati Manzari

发表信息：: Computers in Biology and Medicine (2026)
DOI：: 10.1016/j.compbiomed.2025.111347

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

Med-VCD通过视觉对比解码方法提升医疗大视觉语言模型的事实准确性与幻觉准确性，减少幻觉输出并提高推理效率。

摘要：

大型视觉-语言模型（LVLMs）如今在医疗应用中占据核心地位，例如医学视觉问答和影像报告生成。然而，这些模型仍容易产生看似合理但实际上错误的幻觉输出。在自然图像领域，已有几种解码策略被提出，通过强化视觉证据来缓解幻觉，但大多数依赖于二次解码或回滚程序，这会显著减慢推理过程。此外，现有解决方案往往领域特定，可能会在模态之间或生成内容与真实内容之间引入不一致。我们引入了Med-VCD，一种稀疏的视觉对比解码方法，能够在不增加二次解码时间开销的情况下缓解医疗LVLMs中的幻觉问题。Med-VCD采用了一种新的令牌稀疏化策略，能够在运行时选择视觉信息丰富的令牌，修剪冗余同时保留关键视觉上下文，从而在效率与可靠性之间取得平衡。在八个医疗数据集上的评估，涵盖眼科、放射学和病理学任务的视觉问答、报告生成和专用幻觉基准测试，结果显示，Med-VCD将事实准确性提高了平均13%，并且相对于基线医疗LVLMs，幻觉准确性提高了6%。

英文摘要：

Large vision-language models (LVLMs) are now central to healthcare applications such as medical visual question answering and imaging report generation. Yet, these models remain vulnerable to hallucination outputs that appear plausible but are in fact incorrect. In the natural image domain, several decoding strategies have been proposed to mitigate hallucinations by reinforcing visual evidence, but most rely on secondary decoding or rollback procedures that substantially slow inference. Moreover, existing solutions are often domain-specific and may introduce misalignment between modalities or between generated and ground-truth content. We introduce Med-VCD, a sparse visual-contrastive decoding method that mitigates hallucinations in medical LVLMs without the time overhead of secondary decoding. Med-VCD incorporates a novel token-sparsification strategy that selects visually informed tokens on the fly, trimming redundancy while retaining critical visual context and thus balancing efficiency with reliability. Evaluations on eight medical datasets, spanning ophthalmology, radiology, and pathology tasks in visual question answering, report generation, and dedicated hallucination benchmarks, show that Med-VCD raises factual accuracy by an average of 13\% and improves hallucination accuracy by 6\% relative to baseline medical LVLMs.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。