注意力空间对比引导用于高效缓解大视觉-语言模型中的幻觉
Attention-space Contrastive Guidance for Efficient Hallucination Mitigation in LVLMs
AI总结：
本文提出Attention-space Contrastive Guidance方法，通过对比引导生成更视觉一致且语义忠实的文本，实验表明在CHAIR和POPE数据集上提升了忠实度并降低了延迟。
摘要：
大视觉-语言模型（LVLMs）中的幻觉常常出现在语言先验主导视觉证据时，导致物体误识别和视觉不一致的描述。本文将幻觉缓解视为对比引导，通过在自注意力层中直接操作，构建图像条件和近似文本-only注意力路径，利用掩码基的替代方案并应用轻量级正交投影以抑制与文本-only路径对齐的组件，从而在不训练的情况下提高生成文本的视觉一致性。实验表明，ACG在CHAIR和POPE数据集上比现有无训练基线更有效，同时保持描述质量，并将延迟降低至多步对比解码方法的两倍。
英文摘要：
Hallucinations in large vision--language models (LVLMs) often arise when language priors dominate over visual evidence, leading to object misidentification and visually inconsistent descriptions. We address this problem by framing hallucination mitigation as contrastive guidance that steers generation toward visually grounded and semantically faithful text. We propose Attention-space Contrastive Guidance (ACG), a training-free, single-pass method that operates directly in self-attention layers, where hallucination-inducing cross-modal biases emerge. ACG constructs both image-conditioned and approximate text-only attention paths within a single forward pass, enabling efficient guidance before errors accumulate at the output layer. Because this masking-based surrogate can introduce approximation bias, we further apply a lightweight orthogonal projection that suppresses components aligned with the text-only path, yielding a more visually grounded correction. Experiments on CHAIR and POPE show that ACG improves faithfulness over existing training-free baselines while maintaining caption quality, reducing latency by up to $2\times$ compared to multi-pass contrastive decoding methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
