潜在推理模型的并行测试时缩放
Parallel Test-Time Scaling for Latent Reasoning Models
AI总结：
本文提出通过改进采样和聚合方法，使潜在推理模型能有效利用并行测试时缩放，提升连续空间中的推理能力。
摘要：
平行测试时缩放（TTS）是增强大型语言模型（LLMs）的关键方法，通常通过并行采样多个基于token的思维链并在投票或搜索中聚合结果。最近在潜在推理中，中间推理在连续向量空间中展开，提供了一种比显式思维链更高效的替代方案，但潜在模型是否能同样受益于并行TTS仍存疑问，主要由于连续空间中缺乏采样机制，以及缺乏用于高级轨迹聚合的概率信号。本文通过解决上述问题，使潜在推理模型能够实现并行TTS。在采样方面，我们引入了两种基于不确定性的随机策略：蒙特卡洛 Dropout 和加性高斯噪声。在聚合方面，我们设计了一个通过逐步对比目标训练的潜在奖励模型（LatentRM），以评分和引导潜在推理。广泛的实验和可视化分析显示，这两种采样策略能够随着计算资源的增加而有效扩展，并表现出不同的探索动态，而 LatentRM 能够实现有效的轨迹选择。共同，我们的探索为连续空间中的可扩展推理开辟了新方向。
英文摘要：
Parallel test-time scaling (TTS) is a pivotal approach for enhancing large language models (LLMs), typically by sampling multiple token-based chains-of-thought in parallel and aggregating outcomes through voting or search. Recent advances in latent reasoning, where intermediate reasoning unfolds in continuous vector spaces, offer a more efficient alternative to explicit Chain-of-Thought, yet whether such latent models can similarly benefit from parallel TTS remains open, mainly due to the absence of sampling mechanisms in continuous space, and the lack of probabilistic signals for advanced trajectory aggregation. This work enables parallel TTS for latent reasoning models by addressing the above issues. For sampling, we introduce two uncertainty-inspired stochastic strategies: Monte Carlo Dropout and Additive Gaussian Noise. For aggregation, we design a Latent Reward Model (LatentRM) trained with step-wise contrastive objective to score and guide latent reasoning. Extensive experiments and visualization analyses show that both sampling strategies scale effectively with compute and exhibit distinct exploration dynamics, while LatentRM enables effective trajectory selection. Together, our explorations open a new direction for scalable inference in continuous spaces. Code and checkpoints released at https://github.com/ModalityDance/LatentTTS
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
