arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2512.01183cs.CLcs.AI
收藏

TempPerturb-Eval: 关于RAG鲁棒性中内部温度与外部扰动的联合影响

TempPerturb-Eval: On the Joint Effects of Internal Temperature and External Perturbations in RAG Robustness

Yongxin Zhou, Philippe Mulhem, Didier Schwab

AI总结:

本文研究了RAG系统中内部温度与外部扰动的交互影响,提出分析框架并展示实验结果,揭示高温度设置加剧扰动敏感性,提出评估基准和调参指南。

摘要:

Retrieval-Augmented Generation (RAG)系统的评估通常孤立考察检索质量和生成参数如温度,忽视其交互作用。本文系统研究了文本扰动(模拟噪声检索)与温度设置在多个LLM运行中的交互作用。我们提出一个全面的RAG扰动-温度分析框架,通过不同温度设置对检索文档施加三种不同扰动类型。在HotpotQA上进行大量实验,使用开源和专有LLM,证明性能下降呈现不同模式:高温度设置持续放大对扰动的脆弱性,某些扰动类型在温度范围内表现出非线性敏感性。本文的贡献包括:(1) 一个评估RAG鲁棒性的诊断基准,(2) 一个量化扰动-温度交互的分析框架,(3) 在噪声检索条件下模型选择和参数调优的实用指南。

英文摘要:

The evaluation of Retrieval-Augmented Generation (RAG) systems typically examines retrieval quality and generation parameters like temperature in isolation, overlooking their interaction. This work presents a systematic investigation of how text perturbations (simulating noisy retrieval) interact with temperature settings across multiple LLM runs. We propose a comprehensive RAG Perturbation-Temperature Analysis Framework that subjects retrieved documents to three distinct perturbation types across varying temperature settings. Through extensive experiments on HotpotQA with both open-source and proprietary LLMs, we demonstrate that performance degradation follows distinct patterns: high-temperature settings consistently amplify vulnerability to perturbations, while certain perturbation types exhibit non-linear sensitivity across the temperature range. Our work yields three key contributions: (1) a diagnostic benchmark for assessing RAG robustness, (2) an analytical framework for quantifying perturbation-temperature interactions, and (3) practical guidelines for model selection and parameter tuning under noisy retrieval conditions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑