先见后议:用视觉证据对齐多智能体共识
Seeing Before Agreeing: Aligning Multi-Agent Consensus with Visual Evidence
AI总结:
提出EAGLE框架,通过显式暴露各智能体的视觉证据区域并相互验证,实现无需训练的多智能体视觉问答协作,提升共识可靠性。
摘要:
视觉语言模型(VLM)在视觉问答(VQA)上取得了强劲性能。为了减轻个体幻觉和盲点,通过多智能体协作聚合不同视角已成为一种有前景的范式。虽然这种方法在文本问答中取得了巨大成功,但其在多模态领域的潜力仍未充分探索。现有的多智能体VQA方法主要采用以文本为中心的协议,专注于文本讨论而忽略视觉信息的对齐。在这项工作中,我们揭示了一个关键见解:答案级别的共识对于可靠的多智能体VQA是不够的; extit{对齐的视觉证据}——智能体所依赖的图像区域的共享支持——对于可信的共识至关重要。为了利用这一见解,我们提出了EAGLE( extbf{E}vidence- extbf{A}ligned extbf{G}rounded mu extbf{L}ti-agent r extbf{E}asoning),一个无需训练的以证据为中心的框架,用于协调多个VLM智能体。EAGLE显式暴露每个智能体的定位区域作为视觉证据,允许对证据进行相互验证,并使用证据一致性指导最终决策。在六个VQA基准上的实验表明,EAGLE在跨领域实现了最佳平均性能,同时保持轻量、可解释且易于部署。
英文摘要:
Vision-language models (VLMs) have achieved strong performance on visual question answering (VQA). To mitigate individual hallucinations and blind spots, aggregating diverse perspectives via multi-agent collaboration has emerged as a promising paradigm. While this approach has shown great success in textual QA, its potential in the multimodal domain remains under-explored. Existing multi-agent VQA methods predominantly adapt text-centric protocols, focusing on textual discussions while ignoring the alignment of visual information. In this work, we reveal a key insight: answer-level agreement is insufficient for reliable multi-agent VQA; \textit{aligned visual evidence} -- shared support from the image regions agents rely on -- is essential for trustworthy consensus. To leverage this insight, we propose EAGLE (\textbf{E}vidence-\textbf{A}ligned \textbf{G}rounded mu\textbf{L}ti-agent r\textbf{E}asoning), a training-free evidence-centered framework for coordinating multiple VLM agents. EAGLE explicitly exposes each agent's grounding regions as visual evidence, enables mutual verification over the evidence, and uses evidence consistency to guide final decision-making. Experiments on six VQA benchmarks show that EAGLE achieves best average performance across domains while remaining lightweight, interpretable, and practical for deployment.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。