arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2606.16620cs.LGcs.AI
收藏

熵门控潜在递归

Entropy-Gated Latent Recursion

Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)

Soham Bhattacharjee, Dushyant Singh Chauhan, Salem Lahlou, Martin Takac, Nils Lukas

AI总结:

提出熵门控潜在递归(EGLR),通过在高不确定性token处递归应用冻结模型顶层解码器,构建与温度采样正交的确定性采样轴,扩展推理时缩放空间,在数学推理任务中显著提升性能。

摘要:

推理时缩放已成为改进语言模型推理能力的主要手段,但现有方法的展开多样性仅来源于单一来源:随机token级采样。我们认为这种单轴采样空间本质上是受限的,并识别出第二个完全确定且互补的轴:在冻结模型的顶层解码器层在高不确定性token处递归重新应用的层跨度$L$。不同的$L$选择会产生不同的展开,解决不同的问题子集,且无需随机性。我们通过熵门控潜在递归(EGLR)实例化这一轴,这是一种无需训练的解码过程,它重新应用顶层$L$层最多$K_{\max}$次迭代,直到下一个token分布收敛。结合$T$个温度采样,EGLR将单轴随机展开池转变为$L\times T$笛卡尔采样空间,且几乎不增加每次展开的成本。我们在8个指令微调模型和6个数学推理基准上表征了这一空间,并表明$L$轴与温度确实互补:在MATH-500上使用Qwen2.5-3B-Instruct时,联合$L\times T$预言机达到91.6%,比仅温度预言机(83.4%)高出8.2个百分点,比仅层预言机(81.2%)高出10.4个百分点,证实两个轴捕获了真正互补的问题。扩展的展开池为任何下游过程(包括自一致性、带验证器的最佳$N$选择和组相对RL训练(GRPO))提供了更丰富的每个提示候选,开辟了不依赖随机噪声的推理时缩放新方向。

英文摘要:

Inference-time scaling has become the dominant lever for improving language-model reasoning, but existing methods derive rollout diversity from a single source: stochastic token-level sampling. We argue that this single-axis sampling space is fundamentally limiting, and identify a second, fully deterministic and complementary axis: the layer span $L$ at which a frozen model's top decoder layers are recursively re-applied at high-uncertainty tokens. Different choices of $L$ produce distinct rollouts that solve different subsets of problems, with no stochasticity. We instantiate this axis through Entropy-Gated Latent Recursion (EGLR), a training-free decoding procedure that re-applies the top-$L$ layers for at most $K_{\max}$ iterations until the next-token distribution converges. Combined with $T$ temperature samples, EGLR turns a single-axis stochastic rollout pool into an $L\times T$ Cartesian sampling space at almost the same per-rollout cost. We characterize this space across $8$ instruction-tuned models and $6$ math reasoning benchmarks, and show that the $L$-axis is genuinely complementary to temperature: on MATH-500 with Qwen2.5-3B-Instruct, the joint $L\times T$ oracle reaches $91.6\%$, $+8.2$ percentage points beyond the temperature-only oracle ($83.4\%$) and $+10.4$ points beyond the layer-only oracle ($81.2\%$), confirming that the two axes capture genuinely complementary problems. The expanded rollout pool provides richer per-prompt candidates for any downstream procedure that consumes rollouts, including self-consistency, best-of-$N$ with verifiers, and group-relative RL training (GRPO), opening a new direction for inference-time scaling that does not rely on stochastic noise.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑