AVAM: 嵌入多模态大语言模型的通用无训练自适应视觉锚定用于多图像问答
AVAM: Universal Training-free Adaptive Visual Anchoring Embedded into Multimodal Large Language Model for Multi-image Question Answering
AI总结：
针对多图像问答中视觉冗余问题，提出可嵌入多模态大语言模型的无训练自适应视觉锚定策略与协同解码机制，提升了多种模型的准确性和效率。
摘要：
多模态大语言模型（MLLMs）的发展推动了视觉问答（VQA）领域的重大进步，从单图像问答演进至多图像问答（MVQA）。然而，MVQA中图像数量的增加不可避免地引入了大量与问答无关的视觉冗余，对准确性和效率产生负面影响。为解决这一问题，现有方法在控制压缩视觉token数量方面缺乏灵活性，且易生成离散视觉片段，阻碍MLLMs整体理解图像的能力。本文提出一种简单且通用的自适应视觉锚定策略，可无缝集成到现有MLLMs中，通过自适应压缩实现显著的准确性提升。同时，为平衡全局和压缩视觉输入的结果，进一步引入新型协同解码机制以实现最优性能。大量实验验证了该方法的有效性，显示其在多种MLLMs上均实现了持续的性能提升。代码将公开可用。
英文摘要：
The advancement of Multimodal Large Language Models (MLLMs) has driven significant progress in Visual Question Answering (VQA), evolving from Single to Multi Image VQA (MVQA). However, the increased number of images in MVQA inevitably introduces substantial visual redundancy that is irrelevant to question answering, negatively impacting both accuracy and efficiency. To address this issue, existing methods lack flexibility in controlling the number of compressed visual tokens and tend to produce discrete visual fragments, which hinder MLLMs' ability to comprehend images holistically. In this paper, we propose a straightforward yet universal Adaptive Visual Anchoring strategy, which can be seamlessly integrated into existing MLLMs, offering significant accuracy improvements through adaptive compression. Meanwhile, to balance the results derived from both global and compressed visual input, we further introduce a novel collaborative decoding mechanism, enabling optimal performance. Extensive experiments validate the effectiveness of our method, demonstrating consistent performance improvements across various MLLMs. The code will be publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
