arXiv 2604.245642026-04-30cs.CLcs.IRcs.ITmath.IT

MEG-RAG: 多模态证据 grounding 的量化研究以提升 RAG 中的证据选择

MEG-RAG: Quantifying Multi-modal Evidence Grounding for Evidence Selection in RAG

Institute for Clarity in Documentation（清晰文档研究所）；Inria Paris-Rocquencourt（巴黎- Rocquencourt 国家信息与自动化研究所）；Rajiv Gandhi University（拉吉夫·甘地大学）；Tsinghua University（清华大学）；Palmer Research Laboratories（帕勒尔研究实验室）；Zhejiang University（浙江大学）；Peking University（北京大学）；University of New South Wales（新南威尔士大学）；Macquarie University（麦考瑞大学）；CSIRO’s Data61（CSIRO 数据61）

Xihang Wang, Zihan Wang, Chengkai Huang, Quan Z. Sheng, Lina Yao

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出 MEG-RAG 框架，通过语义感知的多模态证据 grounding 方法提升 RAG 中的证据选择准确性与多模态一致性。

摘要：

多模态检索增强生成（MRAG）旨在解决多模态大语言模型（MLLMs）的关键局限，如幻觉和过时知识。然而，当前 MRAG 系统难以区分检索的多模态数据是否真正支持答案的语义核心或仅提供表面相关性。现有指标常依赖启发式位置基于的置信度，无法捕捉多模态实体的信息密度。为此，我们提出多模态证据 grounding（MEG），一种语义感知的度量方法，量化检索证据的贡献。不同于标准置信度度量，MEG 利用语义确定性锚定，聚焦于高-IDF 信息承载的词元，更准确捕捉答案的语义核心。基于 MEG，我们引入 MEG-RAG 框架，训练一个多模态重排序器，将检索证据与真实答案的语义锚点对齐。通过基于语义 grounding 优先选择高价值内容而非词元概率分布，MEG-RAG 提高了生成输出的准确性和多模态一致性。在 M$^2$RAG 基准上的广泛实验表明，MEG-RAG 一致优于强基线，并在不同教师模型上展示了稳健的泛化能力。

英文摘要：

Multimodal Retrieval-Augmented Generation (MRAG) addresses key limitations of Multimodal Large Language Models (MLLMs), such as hallucination and outdated knowledge. However, current MRAG systems struggle to distinguish whether retrieved multimodal data truly supports the semantic core of an answer or merely provides superficial relevance. Existing metrics often rely on heuristic position-based confidence, which fails to capture the informational density of multimodal entities. To address this, we propose Multi-modal Evidence Grounding (MEG), a semantic-aware metric that quantifies the contribution of retrieved evidence. Unlike standard confidence measures, MEG utilizes Semantic Certainty Anchoring, focusing on high-IDF information-bearing tokens that better capture the semantic core of the answer. Building on MEG, we introduce MEG-RAG, a framework that trains a multimodal reranker to align retrieved evidence with the semantic anchors of the ground truth. By prioritizing high-value content based on semantic grounding rather than token probability distributions, MEG-RAG improves the accuracy and multimodal consistency of generated outputs. Extensive experiments on the M$^2$RAG benchmark show that MEG-RAG consistently outperforms strong baselines and demonstrates robust generalization across different teacher models.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。