MAVIS：多模态来源归因的长形式视觉问答基准
MAVIS: A Benchmark for Multimodal Source Attribution in Long-form Visual Question Answering
AI总结：
MAVIS基准旨在评估多模态来源归因系统，通过多模态文档检索和长形式答案生成，揭示多模态设置下信息量、 groundedness 和流畅性之间的权衡与改进方向。
摘要：
来源归因旨在通过为每个陈述添加参考来增强AI生成答案的可靠性，帮助用户验证提供的答案。然而，现有研究主要集中在纯文本场景， largely 忽略了多模态的作用。我们引入MAVIS，第一个评估多模态来源归因系统的基准，该系统能够理解视觉问题背后的用户意图，检索多模态证据，并生成带有引用的长形式答案。我们的数据集包含157,000个视觉问答实例，其中每个答案都标注有事实层面的引用，指向多模态文档。我们开发了细粒度的自动指标，沿信息量、 groundedness 和流畅性三个维度进行评估，并展示了其与人类判断的强相关性。我们的主要发现有三项：（1）具有多模态RAG的LVLM生成的信息量和流畅性更高的答案，但它们在图像文档上的 groundedness 比文本文档弱，这种差距在多模态设置中更加明显。（2）在相同多模态文档下，不同提示方法之间存在信息量与 groundedness 之间的权衡。（3）我们提出的方法强调减少上下文偏见在解释图像文档中的影响是未来研究的关键方向。
英文摘要：
Source attribution aims to enhance the reliability of AI-generated answers by including references for each statement, helping users validate the provided answers. However, existing work has primarily focused on text-only scenario and largely overlooked the role of multimodality. We introduce MAVIS, the first benchmark designed to evaluate multimodal source attribution systems that understand user intent behind visual questions, retrieve multimodal evidence, and generate long-form answers with citations. Our dataset comprises 157K visual QA instances, where each answer is annotated with fact-level citations referring to multimodal documents. We develop fine-grained automatic metrics along three dimensions of informativeness, groundedness, and fluency, and demonstrate their strong correlation with human judgments. Our key findings are threefold: (1) LVLMs with multimodal RAG generate more informative and fluent answers than unimodal RAG, but they exhibit weaker groundedness for image documents than for text documents, a gap amplified in multimodal settings. (2) Given the same multimodal documents, there is a trade-off between informativeness and groundedness across different prompting methods. (3) Our proposed method highlights mitigating contextual bias in interpreting image documents as a crucial direction for future research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
