MLDocRAG: 多模态长上下文文档检索增强生成
MLDocRAG: Multimodal Long-Context Document Retrieval Augmented Generation
AI总结：
MLDocRAG通过构建多模态片段-查询图，实现多模态长上下文文档的检索增强生成，提升问答的准确性和连贯性。
摘要：
理解由段落、图表和表格等多模态片段组成的多模态长上下文文档具有挑战性，原因包括（1）跨模态异质性，需要在不同模态中定位相关信息，（2）跨页推理，需要在页面间聚合分散的证据。为了解决这些挑战，我们受到启发，采用以查询为中心的公式，将跨模态和跨页信息投影到统一的查询表示空间中，查询作为抽象语义代理，用于异构多模态内容。在本文中，我们提出了一种多模态长上下文文档检索增强生成（MLDocRAG）框架，该框架利用多模态片段-查询图（MCQG）来围绕语义丰富且可回答的查询组织多模态文档内容。MCQG通过多模态文档扩展过程生成细粒度查询，从异构文档片段中生成，并将它们链接到其对应的内容跨模态和页面。这种基于图的结构使选择性、以查询为中心的检索和结构化证据聚合成为可能，从而增强多模态长上下文问答的 grounding 和连贯性。在MMLongBench-Doc和LongDocURL数据集上的实验表明，MLDocRAG在检索质量和回答准确性方面始终优于其他方法，证明了其在多模态长上下文理解中的有效性。
英文摘要：
Understanding multimodal long-context documents that comprise multimodal chunks such as paragraphs, figures, and tables is challenging due to (1) cross-modal heterogeneity to localize relevant information across modalities, (2) cross-page reasoning to aggregate dispersed evidence across pages. To address these challenges, we are motivated to adopt a query-centric formulation that projects cross-modal and cross-page information into a unified query representation space, with queries acting as abstract semantic surrogates for heterogeneous multimodal content. In this paper, we propose a Multimodal Long-Context Document Retrieval Augmented Generation (MLDocRAG) framework that leverages a Multimodal Chunk-Query Graph (MCQG) to organize multimodal document content around semantically rich, answerable queries. MCQG is constructed via a multimodal document expansion process that generates fine-grained queries from heterogeneous document chunks and links them to their corresponding content across modalities and pages. This graph-based structure enables selective, query-centric retrieval and structured evidence aggregation, thereby enhancing grounding and coherence in multimodal long-context question answering. Experiments on datasets MMLongBench-Doc and LongDocURL demonstrate that MLDocRAG consistently improves retrieval quality and answer accuracy, demonstrating its effectiveness for multimodal long-context understanding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
