教大语言模型何时不知:学习事前推理的临时批评
Teaching Large Language Models When Not to Know: Learning Temporal Critique for Ex-Ante Reasoning
AI总结:
本文研究大语言模型在时间截止下的推理失败,提出TCFT框架通过训练模型识别时间泄漏并判断响应的可接受性,实验表明其在减少时间泄漏方面优于提示和监督微调方法。
摘要:
大语言模型(LLMs)在时间截止下推理时常失败:当被提示从较早的时间角度回答时,它们会利用后来才获得的知识。我们通过事前推理的视角研究这一失败,其中模型必须仅依赖截止前可获得的信息。通过系统分析提示层面的干预,我们发现时间泄漏对截止 formulation 和指令位置高度敏感:显式截止陈述优于隐式历史框架,前缀约束比后缀约束更有效减少泄漏。这些发现表明提示可以引导模型进入时间框架,但不赋予其验证响应是否时间可接受的能力。我们进一步认为监督微调不足,因为事前正确性不是答案的固有属性,而是答案与截止之间的关系。为解决这一差距,我们提出TCFT,一种时间批评微调框架,训练模型获取截止感知的时间验证能力。给定查询、截止和候选响应,TCFT教会模型识别截止后泄漏,解释时间边界违规,并判断时间可接受性。在Qwen2.5-7B-Instruct和Qwen2.5-14B-Instruct上的实验表明,TCFT在减少时间泄漏方面优于提示和监督微调基线,分别减少41.89和37.79个百分点。
英文摘要:
Large language models (LLMs) often fail to reason under temporal cutoffs: when prompted to answer from the standpoint of an earlier time, they exploit knowledge that became available only later. We study this failure through the lens of ex-ante reasoning, where a model must rely exclusively on information knowable before a cutoff. Through a systematic analysis of prompt-level interventions, we find that temporal leakage is highly sensitive to cutoff formulation and instruction placement: explicit cutoff statements outperform implicit historical framings, and prefix constraints reduce leakage more effectively than suffix constraints. These findings indicate that prompting can steer models into a temporal frame, but does not endow them with the ability to verify whether a response is temporally admissible. We further argue that supervised fine-tuning is insufficient, since ex-ante correctness is not an intrinsic property of an answer, but a relation between the answer and the cutoff. To address this gap, we propose TCFT, a Temporal Critique Fine-Tuning framework that trains models to acquire cutoff-aware temporal verification. Given a query, a cutoff, and a candidate response, TCFT teaches the model to identify post-cutoff leakage, explain temporal boundary violations, and judge temporal admissibility. Experiments with Qwen2.5-7B-Instruct and Qwen2.5-14B-Instruct show that TCFT consistently outperforms prompting and SFT baselines, reducing average leakage by 41.89 and 37.79 percentage points, respectively.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。