将LLM推理内化：通过发现和重播潜在动作
Internalizing LLM Reasoning via Discovery and Replay of Latent Actions
AI总结：
STIR通过动态潜在轨迹控制实现LLM推理内化，提升准确率并减少token消耗。
摘要：
将链式思维过程内化到隐藏状态已成为一种高度高效的扩展测试时间计算的范式。然而，现有激活引导方法依赖于静态控制向量，无法适应复杂推理任务的非平稳演变。为了解决这一限制，我们提出了STIR（用于内部推理的自蒸馏工具），一个将推理增强重新表述为动态潜在轨迹控制问题的框架。STIR引入了一个协同的三阶段流程：（1）微分内在动作诱导从潜在推理成功中提取出引导原语；（2）稀疏控制基底构建精选出一个紧凑且几何多样的工具库；（3）价值调节轨迹干预通过基于锚点的门控动态注入上下文特定的脉冲。在四个代表性模型上的六个算术和逻辑基准测试中，实验表明STIR在平均准确率上提高了1.9%至7.5%，同时将平均token消耗减少了高达35%。这些发现表明，通过动态潜在轨迹控制可以实现显式链式思维的好处，将推理过程内化以绕过显式生成，同时实现更优的保真度。我们的代码可在https://github.com/sznnzs/LLM-Latent-Action上获取。
英文摘要：
The internalization of chain-of-thought processes into hidden states has emerged as a highly efficient paradigm for scaling test-time compute. However, existing activation steering methods rely on static control vectors that fail to adapt to the non-stationary evolution of complex reasoning tasks. To address this limitation, we propose STIR (Self-Distilled Tools for Internal Reasoning), a framework that reformulates reasoning enhancement as a dynamic latent trajectory control problem. STIR introduces a synergistic three-stage pipeline: (1) differential intrinsic action induction harvests latent reasoning successes to crystallize steering primitives; (2) sparse control basis construction curates a compact, geometrically diverse tool library; and (3) value-modulated trajectory intervention dynamically injects context-specific impulses via anchor-based gating. Extensive experiments on six arithmetic and logical benchmarks across four representative models demonstrate that STIR improves average accuracy by 1.9% to 7.5% while reducing average token consumption by up to 35% compared to vanilla decoding. These findings demonstrate that the benefits of explicit chain-of-thought can be realized through dynamic latent trajectory control, internalizing the reasoning process to bypass the explicit generation while achieving superior fidelity. Our code is available at https://github.com/sznnzs/LLM-Latent-Action.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
