LASER: 通过视觉注意力保持与沉没抑制为LVLMs提供矫正透镜
LASER: A Corrective Lens for LVLMs via Visual Attention Preservation and Sink Suppression
AI总结：
针对大型视觉语言模型在长序列解码中视觉遗忘的问题，提出LASER后训练框架，通过视觉接地奖励和沉没抑制奖励调节注意力轨迹与分布，在8个基准上超越强基线。
摘要：
大型视觉语言模型（LVLMs）展现出强大的推理能力，但在长序列解码过程中会出现视觉遗忘，即注意力逐渐偏离视觉证据。现有方法大多将此视为后期注意力衰减问题，或试图通过启发式提醒或事后注意力提升来缓解。通过系统的实证分析，我们发现视觉遗忘下的性能下降主要由两个被忽视的因素驱动：早期注意力衰减破坏了证据获取，以及注意力集中在部分与任务无关的视觉沉没令牌上。受此启发，我们提出LASER，一个后训练框架，在推理过程中调节视觉注意力轨迹和令牌内注意力分布。技术上，LASER引入了两个互补的奖励：视觉接地奖励，鼓励模型在整个解码过程中保持对语义显著的视觉令牌的注意力；以及沉没抑制奖励，惩罚对视觉沉没令牌的过度注意力集中。这些奖励共同保持了早期接地，同时防止注意力崩溃到无信息区域。在八个基准数据集上的大量实验表明，LASER持续优于强基线，验证了注意力感知训练作为视觉遗忘的有效补救措施。
英文摘要：
Large vision-language models (LVLMs) exhibit strong reasoning ability but suffer from visual forgetting during long-horizon decoding, where attention progressively drifts away from visual evidence. Existing methods largely treat this issue as a late-stage attention decay problem or attempt to mitigate it through heuristic reminders or post-hoc attention lifting. Through systematic empirical analysis, we find that performance degradation under visual forgetting is largely driven by two overlooked factors: early-stage attention decay disrupts evidence acquisition, and attention concentration on a subset of task-irrelevant visual sink tokens. Motivated by these insights, we propose LASER, a post-training framework that regulates both the visual attention trajectory and intra-visual token attention distribution during reasoning. Technically, LASER introduces two complementary rewards: a Visual Grounding Reward, which encourages the model to maintain attention on semantically salient visual tokens throughout decoding, and a Sink Suppression Reward, which penalizes excessive attention concentration on visual sink tokens. Together, these rewards preserve early-stage grounding while preventing attention collapse onto uninformative regions. Extensive experiments on eight benchmark datasets demonstrate that LASER consistently outperforms strong baselines, validating attention-aware training as an effective remedy for visual forgetting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
