CoTFormer：一种推理时具备计算成本预算自适应能力的思维链驱动架构
CoTFormer: A Chain-of-Thought Driven Architecture with Budget-Adaptive Computation Cost at Inference
AI总结：
本研究提出CoTFormer架构，在token层面模拟思维链以提升模型精度，同时结合逐token可变深度的计算自适应机制，实现精度不降前提下的推理计算成本显著降低。
摘要：
将语言模型扩展至更大、更深的规模已带来性能的显著提升。尽管这类模型的规模限制了其在计算受限环境中的应用，但持续开发更大、更深基础模型的竞赛仍在进行。与此同时，无论模型规模如何，任务特定技术始终在实现最优下游性能方面发挥着关键作用。其中一项名为思维链（Chain-of-Thought, CoT）的技术尤为值得关注，正如我们在本研究中指出的，它类似于通过多次复用模型来采用更深的Transformer。然而，计算过往token注意力时的一个关键细微差别，使得CoT不同于简单地多次应用模型。基于这一洞见，我们提出CoTFormer——一种在token层面紧密模拟CoT的新型架构，能让我们获得接近大得多的模型的显著精度提升。虽然应用CoT会引入额外计算成本，但我们通过利用CoTFormer与逐token可变深度的特殊兼容性来弥补这一点。通过一个能自动将计算分配给最有需要的token的计算自适应模型，我们证明可以在不损失任何精度的情况下大幅降低计算成本，并且在保持具有竞争力的精度的同时，还能进一步降低计算成本。
英文摘要：
Scaling language models to larger and deeper sizes has led to significant boosts in performance. Even though the size of these models limits their application in compute-constrained environments, the race to continually develop ever larger and deeper foundational models is underway. At the same time -- regardless of the model size -- task-specific techniques continue to play a pivotal role in achieving optimal downstream performance. One of these techniques, called Chain-of-Thought (CoT), is particularly interesting since, as we point out in this work, it resembles employing a deeper transformer through re-applying the model multiple times. However, a key subtlety in computing the attention of past tokens differentiates CoT from simply applying the model several times. Based on this insight, we propose CoTFormer, a novel architecture which closely mimics CoT at the token level, allowing us to obtain significantly improved accuracies close to much larger models. While applying CoT introduces additional computation costs, we compensate for it by leveraging CoTFormer's special compatibility with token-wise variable depth. Through a compute adaptive model -- which automatically allocates the compute to tokens that need it most -- we show that it is possible to reduce the computation cost significantly without any reduction in accuracy, and with further compute cost reductions possible while maintaining a competitive accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
