发表机构
Northeastern University; Tsinghua University（东北大学; 清华大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究复杂任务中语言模型推理轨迹易偏离上下文的问题，提出REFACT自适应事实重述框架，经两阶段优化，实验证明其能提升长上下文问答等能力，减少令牌消耗，保留更多证据使推理轨迹更优。
AI 中文摘要
大语言模型在复杂任务中越来越依赖长形式推理，但在证据稀疏、有噪声或与参数知识冲突时，其推理轨迹可能偏离所提供的上下文。现有方法存在不足，本文提出REFACT，一种自适应事实重述引用框架，训练模型决定推理步骤何时需要上下文基础以及以何种粒度重述源事实。通过两阶段SFT到RL管道优化，实验表明REFACT提高了长上下文问答和反事实忠实度，减少了令牌消耗，保留更多答案相关证据，使推理轨迹更密集。
英文摘要
Large Language Models (LLMs) increasingly leverage long-form reasoning to solve complex tasks, yet their reasoning processes can deviate from the provided context when evidence is incomplete, noisy, or conflicts with parametric knowledge. Existing grounding approaches either append citations after generation or encourage LLMs to retrieve evidence during reasoning, but they often fail to ensure that cited information is sufficient to support intermediate inferences and final answers. To address this limitation, we propose REFACT, an adaptive fact-restatement citation framework that enables LLMs to determine when contextual grounding is needed and selectively restate source facts at appropriate levels of detail for reliable reasoning. To facilitate adaptive citation during reasoning, REFACT first leverages a teacher LLM to construct high-quality citation-aware reasoning trajectories under diverse context conditions with varying evidence lengths, and then optimizes the student LLM through a two-stage SFT-to-RL framework. Experiments on LongBench, LV-Eval, and ConFiQA demonstrate that REFACT improves long-context question answering and counterfactual faithfulness while substantially reducing the number of reasoning tokens. Further analysis reveals that REFACT achieves higher evidence density by preserving more answer-relevant facts with fewer restatements, producing reasoning traces that are more concise yet better grounded. All code and data will be released via https://github.com/NEUIR/REFACT.
