Argus:基于接地思维链的视觉中心推理
Argus: Vision-Centric Reasoning with Grounded Chain-of-Thought
AI总结:
本文提出Argus模型,通过引入视觉注意力接地机制,将对象中心接地作为视觉思维链信号,有效解决了多模态大语言模型在视觉中心场景下精确视觉聚焦推理的局限,显著提升了多模态推理与指代对象接地任务的性能。
摘要:
多模态大语言模型(MLLMs)的最新进展在视觉语言任务中展现出卓越能力,但在需要精确视觉聚焦以进行准确推理的视觉中心场景中常遇困难。本文引入Argus,通过新的视觉注意力接地机制解决这些局限。该方法采用以对象为中心的接地作为视觉思维链信号,在多模态推理任务中实现更有效的目标条件视觉注意力。在多样基准上的评估表明,Argus在多模态推理任务和指代对象接地任务中均表现优异。广泛分析进一步验证了Argus的各项设计选择,揭示了MLLMs中显式语言引导视觉感兴趣区域参与的有效性,强调了从视觉中心视角推进多模态智能的重要性。项目页:https://yunzeman.github.io/argus/
英文摘要:
Recent advances in multimodal large language models (MLLMs) have demonstrated remarkable capabilities in vision-language tasks, yet they often struggle with vision-centric scenarios where precise visual focus is needed for accurate reasoning. In this paper, we introduce Argus to address these limitations with a new visual attention grounding mechanism. Our approach employs object-centric grounding as visual chain-of-thought signals, enabling more effective goal-conditioned visual attention during multimodal reasoning tasks. Evaluations on diverse benchmarks demonstrate that Argus excels in both multimodal reasoning tasks and referring object grounding tasks. Extensive analysis further validates various design choices of Argus, and reveals the effectiveness of explicit language-guided visual region-of-interest engagement in MLLMs, highlighting the importance of advancing multimodal intelligence from a visual-centric perspective. Project page: https://yunzeman.github.io/argus/
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。