沉默即是黄金：大语言模型能否在时间问答中学会退避？
When Silence Is Golden: Can LLMs Learn to Abstain in Temporal QA and Beyond?
AI总结：
本文探讨了如何通过强化学习训练大语言模型在时间问答中学会退避，发现RL在推理和可靠性方面优于SFT。
摘要：
大型语言模型（LLMs）很少承认不确定性，通常产生流畅但误导性的答案，而不是退避（即拒绝回答）。这一弱点在时间问答中也显而易见，其中模型经常忽略时间敏感的证据，并在不同时间段混杂事实。在本文中，我们提出了首次对训练具有退避能力的LLMs进行时间问答推理的实证研究。现有的方法如校准可能在复杂推理中捕捉不确定性不可靠。我们将其视为可教授的技能，并引入一个结合链式推理（CoT）监督与由退避意识奖励引导的强化学习（RL）的流程。我们的目标是系统分析不同信息类型和训练技术如何影响具有退避行为的LLMs的时间推理。通过广泛的实验研究各种方法，我们发现RL在推理上产生了显著的实证收益：一个由Qwen2.5-1.5B-Instruct初始化的模型在TimeQA-Easy和Hard上的精确匹配度分别超过GPT-4o 3.46%和5.80%。此外，它在不可回答问题上的真正阳性率比纯监督微调（SFT）变体提高了20%。此外，我们的分析显示，SFT导致过度自信并损害可靠性，而RL提高预测准确性但表现出相似的风险。最后，通过比较隐含推理线索（例如原始上下文、时间子上下文、知识图谱）与显式的Co T监督，我们发现隐含信息对具有退避的推理提供有限的益处。我们的研究为如何联合优化退避和推理提供了新的见解，为构建更可靠的LLMs提供了基础。
英文摘要：
Large language models (LLMs) rarely admit uncertainty, often producing fluent but misleading answers, rather than abstaining (i.e., refusing to answer). This weakness is even evident in temporal question answering, where models frequently ignore time-sensitive evidence and conflate facts across different time-periods. In this paper, we present the first empirical study of training LLMs with an abstention ability while reasoning about temporal QA. Existing approaches such as calibration might be unreliable in capturing uncertainty in complex reasoning. We instead frame abstention as a teachable skill and introduce a pipeline that couples Chain-of-Thought (CoT) supervision with Reinforcement Learning (RL) guided by abstention-aware rewards. Our goal is to systematically analyze how different information types and training techniques affect temporal reasoning with abstention behavior in LLMs. Through extensive experiments studying various methods, we find that RL yields strong empirical gains on reasoning: a model initialized by Qwen2.5-1.5B-Instruct surpasses GPT-4o by $3.46\%$ and $5.80\%$ in Exact Match on TimeQA-Easy and Hard, respectively. Moreover, it improves the True Positive rate on unanswerable questions by $20\%$ over a pure supervised fine-tuned (SFT) variant. Beyond performance, our analysis shows that SFT induces overconfidence and harms reliability, while RL improves prediction accuracy but exhibits similar risks. Finally, by comparing implicit reasoning cues (e.g., original context, temporal sub-context, knowledge graphs) with explicit CoT supervision, we find that implicit information provides limited benefit for reasoning with abstention. Our study provides new insights into how abstention and reasoning can be jointly optimized, providing a foundation for building more reliable LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
