检索增强语言模型知道自己何时不知道吗？
Do Retrieval Augmented Language Models Know When They Don't Know?
AI总结：
本研究探讨检索增强语言模型（RALMs）的拒答能力，发现其存在过度拒答问题，通过上下文微调可缓解，但改进的拒答行为不一定提升校准质量，最后提出结合不确定性弃权（不执行）的机制以平衡拒答与正确回答。
摘要：
现有大型语言模型（LLMs）偶尔会生成看似合理但事实错误的响应，即幻觉现象。目前有两种主要方法用于缓解幻觉：检索增强语言模型（RALMs）和拒答后训练。然而，当前研究主要关注它们各自的有效性，却忽视了对RALMs拒答能力的评估。理想情况下，如果RALMs知道自己不知道，它们应该拒绝回答。在本研究中，我们提出了一个基本问题：RALMs知道自己何时不知道吗？具体而言，我们研究了三个问题。首先，RALMs在不同的内部和外部知识状态下是否校准良好？我们考察了多种因素的影响。与预期相反，当所有检索到的文档都不相关时，RALMs仍然倾向于拒答那些它们本可以正确回答的问题。其次，鉴于模型明显的过度拒答行为，我们提出第二个问题：RALM的拒答能力与其校准质量如何对齐？我们的结果表明，通过上下文微调可以缓解过度拒答问题。然而，我们观察到，改进的拒答行为并不一定意味着更好的校准或更高的整体准确性。最后，我们提出：能否将具有拒答意识的RALMs与基于不确定性的回答弃权（不执行）相结合，以缓解过度拒答？我们为经过拒答后训练的RALMs开发了一种简单而有效的拒答机制，通过平衡拒答和正确回答来提高其整体回答质量。我们的研究对影响RALM行为的因素提供了更全面的理解。同时，我们强调，RALMs的不确定性估计仍然是一个值得深入研究的开放问题。
英文摘要：
Existing large language models (LLMs) occasionally generate plausible yet factually incorrect responses, known as hallucinations. Two main approaches have been proposed to mitigate hallucinations: retrieval-augmented language models (RALMs) and refusal post-training. However, current research predominantly focuses on their individual effectiveness while overlooking the evaluation of the refusal capability of RALMs. Ideally, if RALMs know when they do not know, they should refuse to answer.In this study, we ask the fundamental question: Do RALMs know when they don't know? Specifically, we investigate three questions. First, are RALMs well calibrated with respect to different internal and external knowledge states? We examine the influence of various factors. Contrary to expectations, when all retrieved documents are irrelevant, RALMs still tend to refuse questions they could have answered correctly. Next, given the model's pronounced \textbf{over-refusal} behavior, we raise a second question: How does a RALM's refusal ability align with its calibration quality? Our results show that the over-refusal problem can be mitigated through in-context fine-tuning. However, we observe that improved refusal behavior does not necessarily imply better calibration or higher overall accuracy. Finally, we ask: Can we combine refusal-aware RALMs with uncertainty-based answer abstention to mitigate over-refusal? We develop a simple yet effective refusal mechanism for refusal-post-trained RALMs that improves their overall answer quality by balancing refusal and correct answers. Our study provides a more comprehensive understanding of the factors influencing RALM behavior. Meanwhile, we emphasize that uncertainty estimation for RALMs remains an open problem deserving deeper investigation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
