MiRAGE：一种多智能体框架，用于生成多模态多跳问题-答案数据集以评估RAG系统
MiRAGE: A Multiagent Framework for Generating Multimodal Multihop Question-Answer Dataset for RAG Evaluation
AI总结：
MiRAGE通过多智能体框架生成多模态多跳问题-答案数据集，提升RAG系统评估的准确性和复杂性。
摘要：
检索增强生成（RAG）技术向多模态和高风险企业应用的快速发展，已经超出了特定领域评估基准的发展速度。现有数据集往往依赖通用领域语料库或纯文本检索，无法捕捉专业技术文档的复杂性，其中信息是多模态的，推理需要综合分散的证据。我们通过引入MiRAGE，一种用于RAG系统评估的多智能体框架，来填补这一空白。MiRAGE利用一组专门的智能体协作群，生成经过验证的、领域特定的、多模态的、多跳问题-答案数据集。MiRAGE协调一组专门的智能体：递归上下文优化循环来汇总分散的证据，对抗性验证智能体来确保事实基础，以及一个识别专家身份和相关领域以模仿专家认知流程的智能体。在四个不同的领域（法规、金融、定量生物学和新闻）的广泛实证评估中，证明MiRAGE生成的数据集具有显著更高的推理复杂性（平均超过2.3个跳转）和事实可信度。我们的消融研究指出，如果图像的文本描述可用，MiRAGE可以由LLM驱动。视觉定位仍然是一个前沿问题。通过自动化创建反映专有语料库潜在主题结构的黄金标准评估数据集，MiRAGE提供了严格基准下一代信息检索系统的必要基础设施。
英文摘要：
The rapid evolution of Retrieval-Augmented Generation (RAG) toward multimodal, high-stakes enterprise applications has outpaced the development of domain specific evaluation benchmarks. Existing datasets often rely on general-domain corpora or purely textual retrieval, failing to capture the complexity of specialized technical documents where information is inextricably multimodal and reasoning requires synthesizing disjoint evidence. We address this gap by introducing MiRAGE, a Multiagent framework for RAG systems Evaluation, that leverages a collaborative swarm of specialized agents to generate verified, domain-specific, multimodal, and multi-hop Question-Answer datasets. MiRAGE orchestrates a swarm of specialized agents: a recursive context optimization loop to aggregate scattered evidence, an adversarial verifier agent to guarantee factual grounding, and an agent to recognize the expert persona and the relevant domain to mimic expert cognitive workflows. Extensive empirical evaluation across four distinct domains (regulations, finance, quantitative biology, and journalism) demonstrates that MiRAGE generates datasets with significantly higher reasoning complexity (>2.3 average hops) and factual faithfulness. Our ablation studies point that MiRAGE can be powered by LLMs if textual descriptions of the images are available. Visual grounding still remains a frontier. By automating the creation of gold standard evaluation datasets that reflect the latent thematic structure of proprietary corpora, MiRAGE provides the necessary infrastructure to rigorously benchmark the next generation information retrieval systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
