有意识的注视：用于视觉-语言模型中幻觉抑制的自适应注意力机制
Conscious Gaze: Adaptive Attention Mechanisms for Hallucination Mitigation in Vision-Language Models
AI总结：
CG-VLM通过认知需求传感器和聚焦共识诱导模块，在推理时精准干预视觉-语言模型的注意力，有效抑制幻觉并提升性能。
摘要：
大型视觉-语言模型（VLMs）常常表现出文本惯性，即注意力从视觉证据转向语言先验，导致物体幻觉。现有的解码策略仅在输出对数几率层面干预，因此无法纠正内部推理漂移，而最近基于启发式头部抑制或全局转向向量的内部控制方法缺乏原则性的基础。我们引入了有意识注视（CG-VLM），一种无需训练的推理时间框架，将博弈论可解释性转化为可操作的解码控制。基于Harsanyi交互构建的认知需求传感器估计瞬时视觉-文本协同效应，并识别出视觉基础是必要的时刻。在此信号条件下，一个聚焦共识诱导模块在崩溃成文本先验之前，有选择地将中间层注意力重新定向到视觉标记上。CG-VLM在POPE和CHAIR上实现了InstructBLIP、LLaVA、Qwen-VL和mPLUG的最佳性能，同时保持了通用能力，证明了标记级感知能够实现精确、上下文感知的干预，而不会影响基础知识。
英文摘要：
Large Vision-Language Models (VLMs) often exhibit text inertia, where attention drifts from visual evidence toward linguistic priors, resulting in object hallucinations. Existing decoding strategies intervene only at the output logits and thus cannot correct internal reasoning drift, while recent internal-control methods based on heuristic head suppression or global steering vectors lack principled grounding. We introduce Conscious Gaze (CG-VLM), a training-free, inference-time framework that converts game-theoretic interpretability into actionable decoding control. A Cognitive Demand Sensor built on Harsanyi interactions estimates instantaneous vision-text synergy and identifies moments when visual grounding is necessary. Conditioned on this signal, a Focused Consensus Induction module selectively reorients mid-layer attention toward visual tokens before collapse into text priors. CG-VLM achieves state-of-the-art results on POPE and CHAIR across InstructBLIP, LLaVA, Qwen-VL, and mPLUG, while preserving general capabilities, demonstrating that token-level sensing enables precise, context-aware intervention without compromising foundational knowledge.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
