自适应多模态检索增强生成
Self-adaptive Multimodal Retrieval-Augmented Generation
AI总结：
针对传统RAG在多模态任务中检索固定文档导致信息不全或含噪的问题，提出SAM-RAG方法，通过动态过滤相关文档并验证检索与输出质量，在保持高召回率的同时显著提升了多模态任务的检索精度与响应生成表现。
摘要：
传统的检索增强生成（RAG）方法受限于对固定数量检索文档的依赖，常导致信息不完整或含噪，从而损害任务表现。尽管近期的自适应方法缓解了这些问题，但其在复杂真实的多模态任务中应用仍受限。为此，我们提出一种名为自适应多模态检索增强生成（SAM-RAG）的新方法，专为多模态场景设计。SAM-RAG不仅根据输入查询动态过滤相关文档（需要时包含图像描述），还验证检索文档与输出的质量。大量实验表明，SAM-RAG在检索精度和响应生成上均超越现有SOTA方法。进一步的消融与有效性分析显示，SAM-RAG在多模态RAG任务中保持高召回率的同时提升了整体表现。代码见 https://github.com/SAM-RAG/SAM_RAG。
英文摘要：
Traditional Retrieval-Augmented Generation (RAG) methods are limited by their reliance on a fixed number of retrieved documents, often resulting in incomplete or noisy information that undermines task performance. Although recent adaptive approaches alleviated these problems, their application in intricate and real-world multimodal tasks remains limited. To address these, we propose a new approach called Self-adaptive Multimodal Retrieval-Augmented Generation (SAM-RAG), tailored specifically for multimodal contexts. SAM-RAG not only dynamically filters relevant documents based on the input query, including image captions when needed, but also verifies the quality of both the retrieved documents and the output. Extensive experimental results show that SAM-RAG surpasses existing state-of-the-art methods in both retrieval accuracy and response generation. By further ablation experiments and effectiveness analysis, SAM-RAG maintains high recall quality while improving overall task performance in multimodal RAG task. Our codes are available at https://github.com/SAM-RAG/SAM_RAG.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
