少即是多：基于多信号晚期融合的无图多模态RAG

中文摘要 AI 辅助

基于图的检索增强生成（RAG）被广泛用于多模态、跨文档的问答任务。然而，构建语料库级别的图结构成本高昂、查询速度慢且难以维护。我们提出了TrioRAG，一个无图的多模态框架，它整合了来自三个互补信号的证据：问题、锚定图像以及由两者生成的VLM增强查询。每个信号在共享的多向量索引（包含页面文本和页面图像）上独立检索，并通过晚期融合合并结果。此外，我们引入了AutoQA，一个多模态汽车基准测试，其问题基于嘈杂的、来自网络的图像而非干净的文档图像。其问题需要跨手册进行推理。我们将其定位为模型策划的测试平台，而非人工验证的金标准。在三个基准测试中，TrioRAG匹配或超越了基于图的系统，同时降低了总成本并将每次查询的推理速度提升了1.6至2.3倍。通过设计，AutoQA的问题基于语料库外的网络图像。在此设置下，图像检索的文档级召回率仅为19.3%，而基于文本的信号，尤其是VLM增强查询，保持了检索的稳健性。

英文摘要

Graph-based retrieval-augmented generation (RAG) is widely used for multimodal, cross-document question answering. However, building corpus-level graphs is expensive, slow to query, and difficult to maintain. We present TrioRAG, a graph-free multimodal framework that integrates evidence from three complementary signals: the question, the anchor image, and a VLM-enhanced query generated from both. Each signal retrieves independently over a shared multi-vector index of page text and page images, and the results are combined through late fusion. Further, we introduce AutoQA, a multimodal automotive benchmark whose questions are grounded in noisy, web-sourced images rather than clean document-sourced figures. Its questions require reasoning across manuals. We position it as a model-curated testbed rather than a human-validated gold standard. Across three benchmarks, TrioRAG matches or outperforms graph-based systems while reducing total cost and accelerating per-query inference by 1.6-2.3 times. By construction, AutoQA grounds its questions in out-of-corpus web images. In this setting image retrieval reaches only 19.3% document-level recall, while text-derived signals, especially the VLM-enhanced query, keep retrieval robust.

发表机构

• University of Tübingen（图宾根大学）
• Robert Bosch GmbH（罗伯特·博世有限公司）
• University of Stuttgart（斯图加特大学）

机构由 AI 辅助整理，请以论文原文为准。