思维链令牌是计算机程序变量
Chain-of-Thought Tokens are Computer Program Variables
AI总结：
本文通过多位数乘法和动态规划实验发现思维链令牌在LLMs中起到类似计算机程序变量的作用，仅保留存储中间结果的令牌即可维持性能，但该机制存在非预期捷径和计算复杂度限制等潜在缺陷。
摘要：
思维链（CoT）要求大型语言模型（LLMs）在得出最终答案之前生成中间步骤，并已被证明能有效帮助LLMs解决复杂推理任务。然而，CoT的内部机制在很大程度上仍不清楚。本文在两个组合任务上实证研究了CoT令牌在LLMs中的作用：多位数乘法和动态规划。虽然CoT对于解决这些问题是必不可少的，但我们发现仅保留存储中间结果的令牌就能达到相当的性能。此外，我们观察到以另一种潜在形式存储中间结果不会影响模型性能。我们还随机干预CoT中的一些值，并注意到后续的CoT令牌和最终答案会相应地改变。这些发现表明，CoT令牌的功能可能类似于计算机程序中的变量，但存在潜在缺点，如非预期的捷径和令牌之间的计算复杂度限制。代码和数据可在https://github.com/solitaryzero/CoTs_are_Variables获取。
英文摘要：
Chain-of-thoughts (CoT) requires large language models (LLMs) to generate intermediate steps before reaching the final answer, and has been proven effective to help LLMs solve complex reasoning tasks. However, the inner mechanism of CoT still remains largely unclear. In this paper, we empirically study the role of CoT tokens in LLMs on two compositional tasks: multi-digit multiplication and dynamic programming. While CoT is essential for solving these problems, we find that preserving only tokens that store intermediate results would achieve comparable performance. Furthermore, we observe that storing intermediate results in an alternative latent form will not affect model performance. We also randomly intervene some values in CoT, and notice that subsequent CoT tokens and the final answer would change correspondingly. These findings suggest that CoT tokens may function like variables in computer programs but with potential drawbacks like unintended shortcuts and computational complexity limits between tokens. The code and data are available at https://github.com/solitaryzero/CoTs_are_Variables.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
