当更多反而更少：理解大语言模型中思维链的长度
When More is Less: Understanding Chain-of-Thought Length in LLMs
AI总结：
本文通过实证与理论证明，思维链长度与任务准确率呈倒U形关系，最优长度随任务难度增加而增加、随模型能力增强而减小，并据此提出训练与推理优化指南，以提升LLM推理性能。
摘要：
大语言模型（LLMs）采用思维链（CoT）推理来分解复杂问题。虽然较长的思维链通常被认为更优，但本文挑战了这一观念，认为更长并不总是更好。结合来自真实世界观察、受控实验和理论分析的证据，我们证明了任务准确率通常随思维链长度呈倒U形曲线变化，即性能最初提升，但随着思维链步骤数量的增加最终下降。通过受控实验，我们进一步揭示了最优思维链长度的缩放行为：它随任务难度增加而增加，但随模型能力增强而减小，暴露了一种内在的简单性偏好，即能力更强的模型倾向于更短、更高效的思维链推理。这种偏好在强化学习（RL）训练中也很明显，随着模型准确率的提高，它们倾向于更短的思维链。为了深入理解这些动态，我们建立了一个简单的理论模型，正式证明了这些现象，包括最优长度的缩放规律以及强化学习过程中简单性偏好的出现。在此框架的指导下，我们展示了使用最优长度思维链进行训练以及在推理时采用长度感知过滤所带来的显著实际收益。这些发现既为“过度思考”现象提供了原则性的理解，也为思维链校准提供了多种实用指南，使大语言模型能够通过适应任务复杂性和模型能力的自适应思维链实现最优推理性能。
英文摘要：
Large Language Models (LLMs) employ Chain-of-Thought (CoT) reasoning to deconstruct complex problems. While longer CoTs are often presumed superior, this paper challenges that notion, arguing that longer is not always better. Drawing on combined evidence from real-world observations, controlled experiments, and theoretical analysis, we demonstrate that task accuracy typically follows an inverted U-shaped curve with CoT length, where performance initially improves but eventually decreases as the number of CoT steps increases. With controlled experiments, we further uncover the scaling behaviors of the optimal CoT length: it increases with task difficulty but decreases with model capability, exposing an inherent simplicity bias where more capable models favor shorter, more efficient CoT reasoning. This bias is also evident in Reinforcement Learning (RL) training, where models gravitate towards shorter CoTs as their accuracy improves. To have a deep understanding of these dynamics, we establish a simple theoretical model that formally proves these phenomena, including the optimal length's scaling laws and the emergence of simplicity bias during RL. Guided by this framework, we demonstrate significant practical benefits from training with optimally-lengthed CoTs and employing length-aware filtering at inference. These findings offer both a principled understanding of the "overthinking" phenomenon and multiple practical guidelines for CoT calibration, enabling LLMs to achieve optimal reasoning performance with adaptive CoTs tailored to task complexity and model capability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
