arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2603.06665cs.CVcs.AI

更好的眼睛,更好的思考:为何视觉链式推理在医学中失效

Better Eyes, Better Thoughts: Why Vision Chain-of-Thought Fails in Medicine

Department of Computer Science, City University of Hong Kong (Dongguan)(香港城市大学(东莞)计算机科学系);College of Computing and Data Science (CCDS), Nanyang Technological University(南洋理工大学计算与数据科学学院);School of Computer Science, Northwestern Polytechnical University(西北工业大学计算机科学学院)

Yuan Wu, Zongxian Yang, Jiayu Qian, Songpan Gao, Guanxing Chen, Qiankun Li, Yu-An Huang, Zhi-An Huang

AI总结:

本文发现链式推理在医学视觉问答中表现欠佳,提出通过感知锚定和描述 grounding 提升视觉 grounding 和跨模态对齐以改善性能。

摘要:

大型视觉-语言模型(VLMs)在通用领域常受益于链式推理(CoT)提示,但在医学视觉-语言任务中效果尚不明确。本文发现,在医学视觉问答中,CoT在通用和医学专用模型中均常劣于直接回答(DirA)。这归因于医学感知瓶颈:细微的领域特定线索可能削弱视觉 grounding,而Co T可能放大早期感知不确定性而非纠正。为验证这一假设,我们引入两种无需训练的推理时 grounding 干预:(i)通过感兴趣区域线索进行感知锚定;(ii)通过高质量文本指导进行描述 grounding。在多个基准和模型家族中,这些干预提高了准确性,缓解了CoT退化,并在某些设置中逆转了CoT-DirA的倒置。我们的发现表明,可靠的临床VLMs需要稳健的视觉 grounding 和跨模态对齐,而不仅仅是扩展文本驱动的推理链。代码可在https://github.com/TianYin123/Better_Eyes_Better_Thoughts获取。

英文摘要:

Large vision-language models (VLMs) often benefit from chain-of-thought (CoT) prompting in general domains, yet its efficacy in medical vision-language tasks remains underexplored. We report a counter-intuitive trend: on medical visual question answering, CoT frequently underperforms direct answering (DirA) across general-purpose and medical-specific models. We attribute this to a \emph{medical perception bottleneck}: subtle, domain-specific cues can weaken visual grounding, and CoT may compound early perceptual uncertainty rather than correct it. To probe this hypothesis, we introduce two training-free, inference-time grounding interventions: (i) \emph{perception anchoring} via region-of-interest cues and (ii) \emph{description grounding} via high-quality textual guidance. Across multiple benchmarks and model families, these interventions improve accuracy, mitigate CoT degradation, and in several settings reverse the CoT--DirA inversion. Our findings suggest that reliable clinical VLMs require robust visual grounding and cross-modal alignment, beyond extending text-driven reasoning chains. Code is available \href{https://github.com/TianYin123/Better_Eyes_Better_Thoughts}{here}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑