视觉注意力永不消退：面向多模态大语言模型详细图像描述的选择性渐进注意力重校准
Visual Attention Never Fades: Selective Progressive Attention ReCalibration for Detailed Image Captioning in Multimodal Large Language Models
AI总结：
本文提出无需训练的 SPARC 方法，通过选择性渐进重校准视觉注意力，在极小计算开销下同时提升多模态大语言模型详细图像描述的精确率与召回率。
摘要：
详细图像描述对于数据生成和辅助视障人士等任务至关重要。高质量的描述需要在精确率和召回率之间取得平衡，这对当前的多模态大语言模型（MLLMs）而言仍然具有挑战性。在本工作中，我们假设这一局限源于随着响应变长，视觉注意力逐渐减弱且噪声不断增加。为了解决这一问题，我们提出了 SPARC（Selective Progressive Attention ReCalibration，选择性渐进注意力重校准），这是一种无需训练的方法，可在解码过程中增强视觉 token 的贡献。SPARC 建立在三个关键观察之上：（1）增加所有视觉 token 的影响会降低召回率，因此 SPARC 选择性地放大视觉 token；（2）随着描述变长，视觉注意力变得更加嘈杂，因此 SPARC 通过利用跨时间步的注意力差异来识别关键的视觉 token；（3）随着视觉注意力逐渐减弱，SPARC 对其进行强化以保持其影响力。我们的实验结合了自动评估和人工评估，结果表明现有方法以牺牲召回率为代价提高了 MLLMs 的精确率。相比之下，我们提出的方法在计算开销极小的情况下同时提升了精确率和召回率。
英文摘要：
Detailed image captioning is essential for tasks like data generation and aiding visually impaired individuals. High-quality captions require a balance between precision and recall, which remains challenging for current multimodal large language models (MLLMs). In this work, we hypothesize that this limitation stems from weakening and increasingly noisy visual attention as responses lengthen. To address this issue, we propose SPARC (Selective Progressive Attention ReCalibration), a training-free method that enhances the contribution of visual tokens during decoding. SPARC is founded on three key observations: (1) increasing the influence of all visual tokens reduces recall; thus, SPARC selectively amplifies visual tokens; (2) as captions lengthen, visual attention becomes noisier, so SPARC identifies critical visual tokens by leveraging attention differences across time steps; (3) as visual attention gradually weakens, SPARC reinforces it to preserve its influence. Our experiments, incorporating both automated and human evaluations, demonstrate that existing methods improve the precision of MLLMs at the cost of recall. In contrast, our proposed method enhances both precision and recall with minimal computational overhead.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
