控制检索增强生成的风险：一个反事实提示框架
Controlling Risk of Retrieval-augmented Generation: A Counterfactual Prompting Framework
- ICT, CAS（中国科学院信息技术研究所）
- University of Chinese Academy of Sciences（中国科学院大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
针对RAG预测不确定性导致的风险失控问题，本文提出反事实提示框架，通过改变检索质量与利用方式评估模型置信度，并构建带弃权选项的基准和风险指标验证其有效性。
AI中文摘要：
检索增强生成（RAG）已成为缓解大型语言模型幻觉问题的一种流行方案。然而，现有关于RAG的研究很少处理预测不确定性问题，即RAG模型的预测不正确的可能性有多大，这导致其在现实应用中存在不可控风险。在本工作中，我们强调风险控制的重要性，即确保RAG模型主动拒绝回答置信度较低的问题。我们的研究识别出影响RAG对其预测置信度的两个关键潜在因素：检索结果的质量，以及这些结果被利用的方式。为引导RAG模型基于这两个潜在因素评估自身置信度，我们开发了一种反事实提示框架，促使模型改变这些因素，并分析其对答案产生的影响。我们还提出了一种基准评测流程，用于收集带有弃权（不执行）选项的答案，从而支持一系列实验。在评估方面，我们引入了若干与风险相关的指标，实验结果证明了本方法的有效性。我们的代码和基准数据集可在https://github.com/ict-bigdatalab/RC-RAG获取。
英文摘要：
Retrieval-augmented generation (RAG) has emerged as a popular solution to mitigate the hallucination issues of large language models. However, existing studies on RAG seldom address the issue of predictive uncertainty, i.e., how likely it is that a RAG model's prediction is incorrect, resulting in uncontrollable risks in real-world applications. In this work, we emphasize the importance of risk control, ensuring that RAG models proactively refuse to answer questions with low confidence. Our research identifies two critical latent factors affecting RAG's confidence in its predictions: the quality of the retrieved results and the manner in which these results are utilized. To guide RAG models in assessing their own confidence based on these two latent factors, we develop a counterfactual prompting framework that induces the models to alter these factors and analyzes the effect on their answers. We also introduce a benchmarking procedure to collect answers with the option to abstain, facilitating a series of experiments. For evaluation, we introduce several risk-related metrics and the experimental results demonstrate the effectiveness of our approach. Our code and benchmark dataset are available at https://github.com/ict-bigdatalab/RC-RAG.
