arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2602.11201cs.CL

链式思维推理中忠实度衰减的机制证据

Mechanistic Evidence for Faithfulness Decay in Chain-of-Thought Reasoning

Fordham University(福特汉姆大学);Algoverse AI Research(Algoverse AI研究);Rice University(稻子大学);UC San Diego(圣地亚哥大学);Santa Clara University(圣克拉拉大学);LMU Munich(慕尼黑路德维希-马克西米利安大学)

Donald Ye, Max Loffgren, Om Kotadia, Linus Wong, Jonas Rohweder

AI总结:

提出归一化对数几率差衰减(NLDD)指标,通过破坏推理步骤并测量模型置信度下降,发现链式思维中超过70-85%长度的令牌对最终答案贡献微弱或负面,揭示了忠实度衰减现象。

摘要:

链式思维(CoT)解释被广泛用于解释语言模型如何解决复杂问题,但目前尚不清楚这些逐步解释是否反映了模型实际得出答案的方式,还是仅仅是事后证明。我们提出了归一化对数几率差衰减(NLDD),一种衡量单个推理步骤是否忠实于模型决策过程的指标。我们的方法从解释中破坏单个推理步骤,并测量模型对其答案的置信度下降程度,以确定该步骤是否真正重要。通过标准化这些测量,NLDD能够实现跨不同架构的严格跨模型比较。在三种模型家族上测试句法、逻辑和算术任务,我们发现了一个一致的推理视界(k*),位于链长的70-85%处,超过该点的推理令牌对最终答案几乎没有或只有负面影响。我们还发现,模型可以在完全失败任务的同时编码正确的内部表示。这些结果表明,仅凭准确性并不能揭示模型是否真正通过其链进行推理。NLDD提供了一种衡量CoT何时重要的方法。

英文摘要:

Chain-of-Thought (CoT) explanations are widely used to interpret how language models solve complex problems, yet it remains unclear whether these step-by-step explanations reflect how the model actually reaches its answer, or merely post-hoc justifications. We propose Normalized Logit Difference Decay (NLDD), a metric that measures whether individual reasoning steps are faithful to the model's decision-making process. Our approach corrupts individual reasoning steps from the explanation and measures how much the model's confidence in its answer drops, to determine if a step is truly important. By standardizing these measurements, NLDD enables rigorous cross-model comparison across different architectures. Testing three model families across syntactic, logical, and arithmetic tasks, we discover a consistent Reasoning Horizon (k*) at 70--85% of chain length, beyond which reasoning tokens have little or negative effect on the final answer. We also find that models can encode correct internal representations while completely failing the task. These results show that accuracy alone does not reveal whether a model actually reasons through its chain. NLDD offers a way to measure when CoT matters.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑