更好的眼睛,更好的思考:为何视觉链式推理在医学中失效
Better Eyes, Better Thoughts: Why Vision Chain-of-Thought Fails in Medicine
AI总结:
本文发现链式推理在医学视觉问答中表现欠佳,提出通过感知锚定和描述 grounding 提升视觉 grounding 和跨模态对齐以改善性能。
摘要:
大型视觉-语言模型(VLMs)在通用领域常受益于链式推理(CoT)提示,但在医学视觉-语言任务中效果尚不明确。本文发现,在医学视觉问答中,CoT在通用和医学专用模型中均常劣于直接回答(DirA)。这归因于医学感知瓶颈:细微的领域特定线索可能削弱视觉 grounding,而Co T可能放大早期感知不确定性而非纠正。为验证这一假设,我们引入两种无需训练的推理时 grounding 干预:(i)通过感兴趣区域线索进行感知锚定;(ii)通过高质量文本指导进行描述 grounding。在多个基准和模型家族中,这些干预提高了准确性,缓解了CoT退化,并在某些设置中逆转了CoT-DirA的倒置。我们的发现表明,可靠的临床VLMs需要稳健的视觉 grounding 和跨模态对齐,而不仅仅是扩展文本驱动的推理链。代码可在https://github.com/TianYin123/Better_Eyes_Better_Thoughts获取。
英文摘要:
Large vision-language models (VLMs) often benefit from chain-of-thought (CoT) prompting in general domains, yet its efficacy in medical vision-language tasks remains underexplored. We report a counter-intuitive trend: on medical visual question answering, CoT frequently underperforms direct answering (DirA) across general-purpose and medical-specific models. We attribute this to a \emph{medical perception bottleneck}: subtle, domain-specific cues can weaken visual grounding, and CoT may compound early perceptual uncertainty rather than correct it. To probe this hypothesis, we introduce two training-free, inference-time grounding interventions: (i) \emph{perception anchoring} via region-of-interest cues and (ii) \emph{description grounding} via high-quality textual guidance. Across multiple benchmarks and model families, these interventions improve accuracy, mitigate CoT degradation, and in several settings reverse the CoT--DirA inversion. Our findings suggest that reliable clinical VLMs require robust visual grounding and cross-modal alignment, beyond extending text-driven reasoning chains. Code is available \href{https://github.com/TianYin123/Better_Eyes_Better_Thoughts}{here}.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。