Efficient Agent：优化多模态检索增强生成的规划能力
Efficient Agent: Optimizing Planning Capability for Multimodal Retrieval Augmented Generation
AI总结：
提出E-Agent智能体框架，通过mRAG规划器和工具感知执行序列优化多模态检索增强生成的规划能力，并引入RemPlan基准，在准确率提升13%的同时减少37%冗余搜索。
摘要：
多模态检索增强生成（mRAG）已成为解决多模态大语言模型（MLLMs）在新闻分析和热门话题等现实场景中时效性限制的有前景的方案。然而，现有方法常受限于僵化的检索策略和对视觉信息的利用不足。为弥补这一差距，我们提出了E-Agent，这是一个具有两项关键创新的智能体框架：一个经训练能基于上下文推理动态编排多模态工具的mRAG规划器，以及一个采用工具感知执行序列以实现优化mRAG工作流的任务执行器。E-Agent采用一次性mRAG规划策略，能够高效检索信息，同时最大限度地减少冗余工具调用。为严格评估mRAG系统的规划能力，我们引入了Real-World mRAG Planning（RemPlan）基准。该新基准包含依赖检索和不依赖检索的问题类型，并系统性地标注了每个实例所需的基本检索工具。该基准明确的mRAG规划标注和多样化的问题设计，通过模拟需要动态mRAG决策的现实场景，增强了其实际相关性。在RemPlan和三个现有基准上的实验证明了E-Agent的优越性：相比最先进的mRAG方法准确率提升13%，同时减少冗余搜索37%。
英文摘要：
Multimodal Retrieval-Augmented Generation (mRAG) has emerged as a promising solution to address the temporal limitations of Multimodal Large Language Models (MLLMs) in real-world scenarios like news analysis and trending topics. However, existing approaches often suffer from rigid retrieval strategies and under-utilization of visual information. To bridge this gap, we propose E-Agent, an agent framework featuring two key innovations: a mRAG planner trained to dynamically orchestrate multimodal tools based on contextual reasoning, and a task executor employing tool-aware execution sequencing to implement optimized mRAG workflows. E-Agent adopts a one-time mRAG planning strategy that enables efficient information retrieval while minimizing redundant tool invocations. To rigorously assess the planning capabilities of mRAG systems, we introduce the Real-World mRAG Planning (RemPlan) benchmark. This novel benchmark contains both retrieval-dependent and retrieval-independent question types, systematically annotated with essential retrieval tools required for each instance. The benchmark's explicit mRAG planning annotations and diverse question design enhance its practical relevance by simulating real-world scenarios requiring dynamic mRAG decisions. Experiments across RemPlan and three established benchmarks demonstrate E-Agent's superiority: 13% accuracy gain over state-of-the-art mRAG methods while reducing redundant searches by 37%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
