arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2507.22928cs.CLcs.AI

思维链如何思考?基于稀疏自编码的思维链推理机制可解释性研究

How does Chain of Thought Think? Mechanistic Interpretability of Chain-of-Thought Reasoning with Sparse Autoencoding

Xi Chen, Aske Plaat, Niki van Stein

AI总结:

该研究通过稀疏自编码与激活修补技术,首次从特征层面因果探究思维链(CoT)的忠实性,发现CoT能在高容量大语言模型中诱导更可解释的内部结构,且存在明确的规模效应阈值。

摘要:

思维链(Chain-of-thought, CoT)提示提升了大语言模型在多步任务上的准确率,但生成的“思维”是否反映真实的内部推理过程仍未得到解答。我们开展了首个针对CoT忠实性的特征层面因果研究。结合稀疏自编码器与激活修补技术,我们在Pythia-70M和Pythia-2.8B模型处理GSM8K数学题时,分别在CoT提示和普通(noCoT)提示下提取了单语义特征。将少量CoT推理特征替换到noCoT运行中时,2.8B模型的答案对数概率显著提升,但70M模型无可靠效果,揭示了明确的规模阈值。在更大模型中,CoT还带来了显著更高的激活稀疏度与特征可解释性得分,表明内部计算更具模块化。例如,模型生成正确答案的置信度从1.2提升至4.3。我们提出了修补曲线和随机特征修补基线,证明有用的CoT信息不仅存在于top-K修补位置,还广泛分布。总体而言,我们的结果表明CoT能在高容量大语言模型中诱导出更可解释的内部结构,验证了其作为结构化提示方法的作用。

英文摘要:

Chain-of-thought (CoT) prompting boosts Large Language Models accuracy on multi-step tasks, yet whether the generated "thoughts" reflect the true internal reasoning process is unresolved. We present the first feature-level causal study of CoT faithfulness. Combining sparse autoencoders with activation patching, we extract monosemantic features from Pythia-70M and Pythia-2.8B while they tackle GSM8K math problems under CoT and plain (noCoT) prompting. Swapping a small set of CoT-reasoning features into a noCoT run raises answer log-probabilities significantly in the 2.8B model, but has no reliable effect in 70M, revealing a clear scale threshold. CoT also leads to significantly higher activation sparsity and feature interpretability scores in the larger model, signalling more modular internal computation. For example, the model's confidence in generating correct answers improves from 1.2 to 4.3. We introduce patch-curves and random-feature patching baselines, showing that useful CoT information is not only present in the top-K patches but widely distributed. Overall, our results indicate that CoT can induce more interpretable internal structures in high-capacity LLMs, validating its role as a structured prompting method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑