arXiv 2509.133342025-09-10cs.AIcs.LG

FRIT：使用因果重要性提升思维链忠实性

FRIT: Using Causal Importance to Improve Chain-of-Thought Faithfulness

Algoverse AI Research（Algoverse AI研究院）

Anand Swaroop, Akshat Nallani, Saksham Uboweja, Adiliia Uzdenova, Michael Nguyen, Kevin Zhu, Sunishchal Dev, Ashwinee Panda, Vasu Sharma, Maheep Chaudhary

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出FRIT，一种可扩展的对齐方法，通过干预推理步骤生成合成数据并应用直接偏好优化，训练模型产生因果一致的推理，显著提升了思维链的忠实性和准确率。

摘要：

思维链（CoT）推理已成为提升大型语言模型在复杂任务上性能的强大工具，但近期研究表明，推理步骤通常无法对最终答案产生因果影响，从而导致脆弱且不可信的输出。先前的方法主要集中于衡量忠实性，而系统性地提升忠实性的方法仍然有限。我们提出了通过干预训练实现的忠实推理（Faithful Reasoning via Intervention Training，FRIT），这是一种可扩展的对齐方法，通过从系统性破坏的样本中学习，训练模型产生因果一致的推理。FRIT通过对模型生成的CoT中的单个推理步骤进行干预来生成合成训练数据，创建忠实/不忠实的样本对，以突出推理何时崩溃。随后，我们应用直接偏好优化（Direct Preference Optimization）来教导模型偏好因果一致的推理路径。在Qwen3-8B和Mistral-7B-v0.1上跨事实和符号推理任务进行评估，FRIT使Mistral在GSM8K上的忠实推理提升了3.4个百分点，同时准确率提升了7.6个百分点。我们的方法提供了首个可扩展的、无需监督的方法，用于训练语言模型产生更可靠和可解释的推理，填补了推理性能与可信度之间的关键空白。我们在\href{https://github.com/Anut-py/frit}发布了代码。

英文摘要：

Chain-of-thought (CoT) reasoning has emerged as a powerful tool for improving large language model performance on complex tasks, but recent work shows that reasoning steps often fail to causally influence the final answer, creating brittle and untrustworthy outputs. Prior approaches focus primarily on measuring faithfulness, while methods for systematically improving it remain limited. We introduce Faithful Reasoning via Intervention Training (FRIT), a scalable alignment method that trains models to produce causally consistent reasoning by learning from systematically corrupted examples. FRIT generates synthetic training data by intervening on individual reasoning steps in model-generated CoTs, creating faithful/unfaithful pairs that highlight when reasoning breaks down. We then apply Direct Preference Optimization to teach models to prefer causally consistent reasoning paths. Evaluating on Qwen3-8B and Mistral-7B-v0.1 across factual and symbolic reasoning tasks, FRIT increases faithful reasoning by $3.4$ percentage points for Mistral on GSM8K while improving accuracy by $7.6$ percentage points. Our approach provides the first scalable, supervision-free method for training language models to produce more reliable and interpretable reasoning, addressing a critical gap between reasoning performance and trustworthiness. We release our code at \href{https://github.com/Anut-py/frit}.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。