arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.00296cs.IR
收藏

RAGRouter-Bench:一种自适应RAG路由的数据库和基准测试

RAGRouter-Bench: A Dataset and Benchmark for Adaptive RAG Routing

Rutgers University(罗格斯大学);University of Liverpool(利物浦大学)

Ziqi Wang, Xi Zhu, Shuhang Lin, Haochen Xue, Minghao Guo, Yongfeng Zhang

AI总结:

本文提出RAGRouter-Bench,首个用于自适应RAG路由的数据库和基准测试,通过整合查询-语料兼容性、细粒度语料指标和统一评估协议,系统评估生成质量和资源消耗,验证自适应路由在不同查询-语料上下文中的有效性。

摘要:

检索增强生成(RAG)已演变为具有不同性能轮廓和资源需求的范式家族,使范式选择成为多准则、依赖上下文的决策问题。然而,现有研究主要集中在孤立方法改进或仅查询基准测试,未系统考察RAG范式在多样化查询-语料上下文中的行为及有效性-效率权衡。本文引入RAGRouter-Bench,首个用于自适应RAG路由的数据库和基准测试。基于查询-语料兼容性,该基准整合三种典型查询类型、细粒度语料指标捕捉结构和语义属性,并采用统一协议评估生成质量和资源消耗。随后,我们实现了多个基础大语言模型在所有查询-语料组合中的标准化RAG范式,构建包含定量指标和LLM-as-a-Judge评估的全面基准,以指导上下文感知且成本有效的RAG路由决策。我们进一步将路由制定为依赖上下文的范式选择,并在构建的数据库上基准测试了多种查询-语料路由器。广泛实验表明,查询-语料对中不存在单一范式适用情况,自适应路由在有效性-效率权衡上优于固定范式选择。这些发现确立查询-语料兼容性为自适应RAG路由的核心原则,并将RAGRouter-Bench定位为下一代RAG系统的系统测试平台。

英文摘要:

Retrieval-augmented generation (RAG) has evolved into a family of paradigms with distinct performance profiles and resource demands, turning paradigm selection into a multi-criteria, context-dependent decision problem. Nevertheless, existing studies largely focus on isolated method improvements or query-only benchmarking, without systematically examining how RAG paradigms behave across diverse query-corpus contexts and effectiveness-efficiency trade-offs. In this work, we introduce RAGRouter-Bench, the first dataset and benchmark for adaptive RAG routing. Grounded in query-corpus compatibility, the benchmark integrates three canonical query types, fine-grained corpus indicators capturing structural and semantic properties, and a unified protocol for evaluating both generation quality and resource consumption. Then, we implement standardized RAG paradigms with multiple backbone LLMs across all query-corpus combinations, constructing a comprehensive benchmark with quantitative metrics and LLM-as-a-Judge evaluations to inform context-aware and cost-effective RAG routing decisions. We further formulate routing as context-dependent paradigm selection and benchmark a range of query-corpus routers on the constructed dataset. Extensive experiments demonstrate that no one-size-fits-all paradigm exists across query-corpus pairs, and that adaptive routing yields more favorable effectiveness-efficiency trade-offs than fixed paradigm selection. These findings establish query-corpus compatibility as a central principle for adaptive RAG routing and position RAGRouter-Bench as a systematic testbed for next-generation RAG systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑