M$^3$KG-RAG：多跳多模态知识图谱增强的检索增强生成
M$^3$KG-RAG: Multi-hop Multimodal Knowledge Graph-enhanced Retrieval-Augmented Generation
AI总结：
本文提出M$^3$KG-RAG，通过构建多跳多模态知识图谱并引入GRASP机制，提升多模态大语言模型的推理深度和回答准确性。
摘要：
检索增强生成（RAG）近年来已被扩展到多模态领域，将多模态大语言模型（MLLMs）与大量外部知识库如多模态知识图谱（MMKGs）连接起来。尽管取得了近期的成功，但在音频-视觉领域，多模态RAG仍然具有挑战性，原因包括1）现有MMKGs的模态覆盖和多跳连接有限，以及2）仅基于共享多模态嵌入空间的相似性进行检索，无法过滤掉不相关或冗余的知识。为了解决这些限制，我们提出了M$^3$KG-RAG，一种多跳多模态知识图谱增强的RAG，能够从MMKGs中检索与查询对齐的音频-视觉知识，从而提高MLLMs的推理深度和回答的准确性。具体而言，我们设计了一个轻量级的多代理流程来构建多跳MMKG（M$^3$KG），其中包含上下文丰富的多模态实体三元组，能够根据输入查询进行模态层面的检索。此外，我们引入了GRASP（Grounded Retrieval And Selective Pruning），确保实体与查询的精确接地，评估回答支持的相关性，并修剪冗余上下文，只保留对响应生成至关重要的知识。在多样化的多模态基准测试中，M$^3$KG-RAG显著提高了MLLMs在多模态推理和接地方面的性能。项目网站：https://kuai-lab.github.io/cvpr2026m3kgrag/
英文摘要：
Retrieval-Augmented Generation (RAG) has recently been extended to multimodal settings, connecting multimodal large language models (MLLMs) with vast corpora of external knowledge such as multimodal knowledge graphs (MMKGs). Despite their recent success, multimodal RAG in the audio-visual domain remains challenging due to 1) limited modality coverage and multi-hop connectivity of existing MMKGs, and 2) retrieval based solely on similarity in a shared multimodal embedding space, which fails to filter out off-topic or redundant knowledge. To address these limitations, we propose M$^3$KG-RAG, a Multi-hop Multimodal Knowledge Graph-enhanced RAG that retrieves query-aligned audio-visual knowledge from MMKGs, improving reasoning depth and answer faithfulness in MLLMs. Specifically, we devise a lightweight multi-agent pipeline to construct multi-hop MMKG (M$^3$KG), which contains context-enriched triplets of multimodal entities, enabling modality-wise retrieval based on input queries. Furthermore, we introduce GRASP (Grounded Retrieval And Selective Pruning), which ensures precise entity grounding to the query, evaluates answer-supporting relevance, and prunes redundant context to retain only knowledge essential for response generation. Extensive experiments across diverse multimodal benchmarks demonstrate that M$^3$KG-RAG significantly enhances MLLMs' multimodal reasoning and grounding over existing approaches. Project website: https://kuai-lab.github.io/cvpr2026m3kgrag/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
