The evaluation of Retrieval-Augmented Generation (RAG) systems typically examines retrieval quality and generation parameters like temperature in isolation, overlooking their interaction. This work presents a systematic investigation of how text perturbations (simulating noisy retrieval) interact with temperature settings across multiple LLM runs. We propose a comprehensive RAG Perturbation-Temperature Analysis Framework that subjects retrieved documents to three distinct perturbation types across varying temperature settings. Through extensive experiments on HotpotQA with both open-source and proprietary LLMs, we demonstrate that performance degradation follows distinct patterns: high-temperature settings consistently amplify vulnerability to perturbations, while certain perturbation types exhibit non-linear sensitivity across the temperature range. Our work yields three key contributions: (1) a diagnostic benchmark for assessing RAG robustness, (2) an analytical framework for quantifying perturbation-temperature interactions, and (3) practical guidelines for model selection and parameter tuning under noisy retrieval conditions.
Rethinking All Evidence: Enhancing Trustworthy Retrieval-Augmented Generation via Conflict-Driven Summarization
University of Chinese Academy of Sciences(中国科学院大学);Chinese Academy of Sciences(中国科学院);National University of Defense Technology(国防科技大学);Chongqing University of Technology(重庆理工大学)
RAG-Star: Enhancing Deliberative Reasoning with Retrieval Augmented Verification and Refinement
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);Wuhan University of Science and Technology(武汉科技大学);National University of Singapore(新加坡国立大学);Northeastern University at Qinhuangdao(东北大学秦皇岛分校);BOSS Zhipin(BOSS直聘)
Model Internals-based Answer Attribution for Trustworthy Retrieval-Augmented Generation
Center for Language and Cognition (CLCG), University of Groningen(格罗宁根大学语言与认知中心(CLCG));Institute for Logic, Language and Computation (ILLC), University of Amsterdam(阿姆斯特丹大学逻辑、语言与计算研究所(ILLC))
RING: Retrieval-Internalized Generation for Continual Large-Scale Knowledge Injection
State Key Laboratory of AI Safety, Institute of Computing Technology, CAS(中国科学院计算技术研究所人工智能安全国家重点实验室);University of Chinese Academy of Sciences(中国科学院大学);Xiaohongshu Inc.(小红书科技有限公司)
Optimizing Multi-Hop Document Retrieval Through Intermediate Representations
School of Software, BNRist, Tsinghua University(软件学院、BNRist、清华大学);Beijing Key Laboratory of Industrial Bigdata System and Application(北京工业大数据系统与应用重点实验室);Gaoling School of Artificial Intelligence, Renmin University of China(甘露人工智能学院、中国人民大学)
Unveiling and Consulting Core Experts in Retrieval-Augmented MoE-based LLMs
School of Computer Science, Fudan University(复旦大学计算机科学技术学院);LightSpeed Studios, Tencent(腾讯光速工作室);Institute of Modern Languages and Linguistics, Fudan University(复旦大学现代语言与语言学研究所);Key Laboratory of Intelligent Information Processing, Fudan University(复旦大学智能信息处理重点实验室);School of Informatics and ELLIS, University of Edinburgh(爱丁堡大学信息学院及欧洲学习与智能系统实验室)
EcoSafeRAG: Efficient Security through Context Analysis in Retrieval-Augmented Generation
Institute of Information Engineering, Chinese Academy of Sciences(信息工程研究所,中国科学院);School of Cybersecurity, University of Chinese Academy of Sciences(网络安全学院,中国科学院大学);State Key Laboratory of Cyberspace Security Defense(网络空间安全防御国家重点实验室)
Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generation
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);School of Artificial Intelligence, Beijing University of Posts and Telecommunications(北京邮电大学人工智能学院)
Generating Leakage-Free Benchmarks for Robust RAG Evaluation
Department of Computer Science, Purdue University(普渡大学计算机科学系);New Jersey Institute of Technology(新泽西理工学院);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Microsoft Research(微软研究院)
CARE-RAG - Clinical Assessment and Reasoning in RAG
Department of Computer Science(计算机科学系);University of Texas at Austin(德克萨斯大学奥斯汀分校);Behavioral Science and Psychiatry(行为科学与精神病学);Department of Statistics(统计学系);University of Michigan(密歇根大学);School of Information(信息学院)
RAG-IGBench: Innovative Evaluation for RAG-based Interleaved Generation in Open-domain Question Answering
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China(认知智能国家重点实验室,中国科学技术大学);Xiaohongshu Inc.(小红书公司);Xi’an Jiaotong University(西安交通大学)
RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework
Tsinghua University(清华大学);Beijing Normal University(北京师范大学);Northeastern University(东北大学);University of Chinese Academy of Sciences(中国科学院大学);ModelBest;University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)