揭示LVLMs中的感知与生成动态：通过验证主导修正缓解幻觉
Revealing Perception and Generation Dynamics in LVLMs: Mitigating Hallucinations via Validated Dominance Correction
AI总结：
本文提出VDC策略，通过验证主导修正缓解LVLMs中的幻觉问题，揭示感知与生成的动态机制。
摘要：
大型视觉-语言模型（LVLMs）展现出了显著的能力，但幻觉问题仍是一个持续的挑战。本文系统分析了LVLMs内部视觉感知和token生成的演变过程，揭示了两个关键模式。首先，感知遵循三阶段GATE过程：早期层进行全局扫描，中间层接近并收紧核心内容，后期层探索补充区域。其次，生成表现出SAD（亚主导积累到主导）模式，幻觉token源于亚主导token的重复积累，这些token缺乏注意力（视觉感知）或前馈网络（内部知识）的支持。基于这些发现，我们设计了VDC（验证主导修正）策略，该策略检测未支持的token并将其替换为验证过的主导token以提高输出可靠性。在多个模型和基准上的广泛实验证实，VDC显著缓解了幻觉问题。
英文摘要：
Large Vision-Language Models (LVLMs) have shown remarkable capabilities, yet hallucinations remain a persistent challenge. This work presents a systematic analysis of the internal evolution of visual perception and token generation in LVLMs, revealing two key patterns. First, perception follows a three-stage GATE process: early layers perform a Global scan, intermediate layers Approach and Tighten on core content, and later layers Explore supplementary regions. Second, generation exhibits an SAD (Subdominant Accumulation to Dominant) pattern, where hallucinated tokens arise from the repeated accumulation of subdominant tokens lacking support from attention (visual perception) or feed-forward network (internal knowledge). Guided by these findings, we devise the VDC (Validated Dominance Correction) strategy, which detects unsupported tokens and replaces them with validated dominant ones to improve output reliability. Extensive experiments across multiple models and benchmarks confirm that VDC substantially mitigates hallucinations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
