Retrieval-augmented generation (RAG) has evolved into a family of paradigms with distinct performance profiles and resource demands, turning paradigm selection into a multi-criteria, context-dependent decision problem. Nevertheless, existing studies largely focus on isolated method improvements or query-only benchmarking, without systematically examining how RAG paradigms behave across diverse query-corpus contexts and effectiveness-efficiency trade-offs. In this work, we introduce RAGRouter-Bench, the first dataset and benchmark for adaptive RAG routing. Grounded in query-corpus compatibility, the benchmark integrates three canonical query types, fine-grained corpus indicators capturing structural and semantic properties, and a unified protocol for evaluating both generation quality and resource consumption. Then, we implement standardized RAG paradigms with multiple backbone LLMs across all query-corpus combinations, constructing a comprehensive benchmark with quantitative metrics and LLM-as-a-Judge evaluations to inform context-aware and cost-effective RAG routing decisions. We further formulate routing as context-dependent paradigm selection and benchmark a range of query-corpus routers on the constructed dataset. Extensive experiments demonstrate that no one-size-fits-all paradigm exists across query-corpus pairs, and that adaptive routing yields more favorable effectiveness-efficiency trade-offs than fixed paradigm selection. These findings establish query-corpus compatibility as a central principle for adaptive RAG routing and position RAGRouter-Bench as a systematic testbed for next-generation RAG systems.
What Should I Cite? A RAG Benchmark for Academic Citation Prediction
Tsinghua University(清华大学);University of Science and Technology of China(中国科学技术大学);Beijing Institute of Technology(北京理工大学);Peking University(北京大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)
RAG vs. GraphRAG: A Systematic Evaluation and Key Insights
Michigan State University(密歇根州立大学);University of Oregon(俄勒冈大学);University of Texas at Arlington(德克萨斯大学阿灵顿分校);Meta Platforms, Inc.(Meta平台公司);IBM T.J. Watson Research Center(IBM托马斯·J·沃森研究中心)
Investigating Retrieval-Augmented Generation Systems on Unanswerable, Uncheatable, Realistic, Multi-hop Queries
Yale University(耶鲁大学);University of Pennsylvania(宾夕法尼亚大学);Human Language Technology Center of Excellence(语言技术中心卓越中心);Johns Hopkins University(约翰霍普金斯大学)
RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignment
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);Beijing Academy of Artificial Intelligence(北京人工智能研究院);Huawei Poisson Lab(华为泊松实验室);Waseda University(早稻田大学)
IRSC: A Zero-shot Evaluation Benchmark for Information Retrieval through Semantic Comprehension in Retrieval-Augmented Generation Scenarios
PengCheng Laboratory(鹏城实验室);Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Southern University of Science and Technology(南方科技大学)
RAG-IGBench: Innovative Evaluation for RAG-based Interleaved Generation in Open-domain Question Answering
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China(认知智能国家重点实验室,中国科学技术大学);Xiaohongshu Inc.(小红书公司);Xi’an Jiaotong University(西安交通大学)
AMAQA: A Metadata-based QA Dataset for RAG Systems
Institute for Informatics and Telematics, National Research Council(信息与电信研究院,国家研究院);Department of Information Engineering, University of Pisa(信息工程系,比萨大学);Department of Computer Science, University of Pisa(计算机科学系,比萨大学)
HawkBench: Investigating Resilience of RAG Methods on Stratified Information-Seeking Tasks
Peking University(北京大学);Beijing Academy of Artificial Intelligence(北京人工智能研究院);Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学人工智能学院);University of Science and Technology of China(中国科学技术大学);The Hong Kong University of Science and Technology(香港科技大学)
SafeRAG: Benchmarking Security in Retrieval-Augmented Generation of Large Language Model
School of Information, Renmin University of China(中国人民大学信息学院);Institute for Advanced Algorithms Research(高级算法研究院);School of Artificial Intelligence, Beihang University(北京航空航天大学人工智能学院)
Benchmarking Retrieval-Augmented Generation for Chemistry
Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校Siebel计算与数据科学学院);National Library of Medicine, National Institutes of Health(美国国立卫生研究院国家医学图书馆)
URAG: A Benchmark for Uncertainty Quantification in Retrieval-Augmented Large Language Models
Uppsala University(乌普萨拉大学);University of Science, VNU-HCM(VNU-HCM大学);Indiana University(印第安纳大学);FPT Software, AI Center(FPT软件人工智能中心);The Pennsylvania State University(宾夕法尼亚州立大学);VNU University of Engineering(VNU工程大学);University of Illinois at Chicago(伊利诺伊大学芝加哥分校)
Toward General Instruction-Following Alignment for Retrieval-Augmented Generation
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);School of Artificial Intelligence, Beijing University of Posts and Telecommunications(北京邮电大学人工智能学院)