AI 中文总结
研究提出MUX方法，将离散推理提炼为连续复用令牌，通过位置相关加权实现无损复用，防止潜在坍缩，能在需搜索问题中并行探索，在多语言模型评估中优于基线，证明无损叠加是实现高效潜在连续推理的充分条件。
AI 中文摘要
语言模型通过用自然语言阐述中间推理步骤来解决复杂问题。虽然有效，但该过程存在计算瓶颈：每个推理步骤仅传达一个子词，且许多步骤用于表达想法而非进行计算。我们提出了MUX，一种基于在潜在空间中将离散推理提炼为连续复用令牌的高带宽和紧凑推理的简单方法。每个潜在令牌经训练以表示离散推理子词跨度的加权线性叠加（复用），这种叠加在构造上是无损的且跨度可完全恢复（解复用）。我们证明简单的位置相关加权，如合适的几何衰减，支持无损复用，进而防止由潜在坍缩引起的捷径行为。我们还表明复用推理可在需要搜索的问题中进行并行探索。在跨越四个语言模型的32个评估设置中，MUX优于强大的潜在推理基线。消融和探测分析进一步表明，学习到的潜在令牌编码了可靠且可解释的推理。我们的结果表明，作为局部学习目标的无损叠加是实现强大且高效的潜在连续推理的充分条件。
英文摘要
Language models solve complex problems by articulating intermediate reasoning steps in natural language. While effective, this process is computationally bottlenecked: each reasoning step conveys only a single subword, and many are spent expressing a thought instead of carrying out computation. We propose MUX, a simple method for high-bandwidth and compact reasoning based on distillation of discrete reasoning into continuous multiplexed tokens in a latent space. Here, each latent token is trained to represent a weighted linear superposition (multiplexing) of a span of discrete reasoning subwords, where this superposition is lossless by construction and the span can be fully recovered (demultiplexing). We prove that simple position-dependent weightings, such as suitable geometric decay, support lossless multiplexing, which in turn prevents shortcut behaviors caused by latent collapse. We further show that multiplexed reasoning can perform parallel exploration in problems that require search. Across 32 evaluation settings spanning four language models, MUX outperforms strong latent reasoning baselines. Ablation and probing analyses further show that the learned latent tokens encode faithful and interpretable reasoning. Our results suggest that lossless superposition as local learning targets constitutes a sufficient condition for achieving strong and efficient latent continuous reasoning.
