模型内部的自我校正：利用层注意力缓解大视觉语言模型中的幻觉
Self-Correction Inside the Model: Leveraging Layer Attention to Mitigate Hallucinations in Large Vision Language Models
AI总结：
本文提出了一种利用层注意力的内部自我校正机制，通过自我细化提高大视觉语言模型的视觉接地性，有效缓解幻觉问题。
摘要：
尽管大视觉语言模型（LVLMs）已取得显著进展，但幻觉——即生成的文本不基于视觉输入——仍是一个挑战。随着LVLMs的增强，之前报告的幻觉模式，如语言偏见和过度思考现象，变得远不一致，使得相应的缓解技术显著效果降低。在本文中，我们引入了一种内部自我校正机制，利用层注意力（ICLA），该机制在生成过程中直接作用于隐藏状态。每一层通过一种对角交叉层注意力机制从所有先前层中选择性地检索信息，从而实现自我细化，而无需任何外部校正信号。在仅引入和训练0.2M和0.1M额外参数于LLaVA1.5-7B和Qwen2.5-VL-7B上后，\ours在多个幻觉基准测试中一致提高了视觉接地性，证明了其对更高级LVLMs的有效性。
英文摘要：
Although Large Vision-Language Models (LVLMs) have made substantial progress, hallucination, where generated text is not grounded in the visual input, remains a challenge. As LVLMs become stronger, previously reported hallucination patterns, such as linguistic bias and overthinking phenomenon, become far less consistent, making the corresponding mitigation techniques substantially less effective. In this paper, we introduce an Internal self-Correction mechanism utilizing Layer Attention (ICLA) that operates directly on hidden states during generation. Each layer selectively retrieves information from all preceding layers through a diagonal cross-layer attention mechanism, enabling self-refinement without any external correction signals. With introducing and training only 0.2M and 0.1M additional parameters on LLaVA1.5-7B and Qwen2.5-VL-7B, \ours consistently improves visual grounding across multiple hallucination benchmarks, demonstrating its effectiveness for more advanced LVLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
