通过模糊序列深度量化链式思维的必要性
Quantifying the Necessity of Chain of Thought through Opaque Serial Depth
AI总结：
本文通过模糊序列深度量化链式思维的必要性，分析了大型语言模型和混合专家模型的推理深度差异。
摘要：
大型语言模型（LLMs）倾向于将推理过程外部化到其链式思维中，使链式思维成为监控的良好目标。这在一定程度上是Transformer架构的固有特征：足够长的序列认知必须通过链式思维（Korbak等人，2025）。我们通过模糊序列深度的概念正式化这一论点，模糊序列深度由不使用可解释中间步骤（如链式思维）所能完成的最长计算的长度给出。基于此形式化，我们计算了Gemma 3模型的模糊序列深度的数值上界，以及标准LLM之外其他架构的渐近结果。我们还开源了一个自动化方法，可以计算任意神经网络的模糊序列深度的上界，并利用它证明混合专家模型可能比密集模型具有更低的深度。总体而言，我们的结果表明，模糊序列深度是理解模型进行未外部化的显著推理潜力的有用工具。
英文摘要：
Large language models (LLMs) tend to externalize their reasoning in their chain of thought, making the chain of thought a good target for monitoring. This is partially an inherent feature of the Transformer architecture: sufficiently long serial cognition must pass through the chain of thought (Korbak et al., 2025). We formalize this argument through the notion of opaque serial depth, given by the length of the longest computation that can be done without the use of interpretable intermediate steps like chain of thought. Given this formalization, we compute numeric upper bounds on the opaque serial depth of Gemma 3 models, as well as asymptotic results for additional architectures beyond standard LLMs. We also open-source an automated method that can calculate upper bounds on the opaque serial depth of arbitrary neural networks, and use it to demonstrate that Mixture-of-Experts models likely have lower depth than dense models. Overall, our results suggest that opaque serial depth is a useful tool for understanding the potential for models to do significant reasoning that is not externalized.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
