ThinkRouter: 通过潜在空间与离散空间之间的路由思维实现高效推理
ThinkRouter: Efficient Reasoning via Routing Thinking between Latent and Discrete Spaces
AI总结:
ThinkRouter通过在低信心时路由至离散空间,高信心时路由至潜在空间,提升推理效率和准确性,平均提高Pass@1 19.70分,减少生成长度15.55%。
摘要:
近期的工作探索了潜在推理,通过将显式推理轨迹替换为潜在空间中的连续表示来提高推理效率,但其效果因场景而异。对模型信心动态的分析表明,以错误答案结束的推理轨迹包含较少的低信心步骤,而以正确答案结束的则相反。同时,我们提出软嵌入可能通过多个低信心推理替代方案聚合并引入和传播噪声,导致不可靠推理轨迹的高信心。受这些观察的启发,提出ThinkRouter,一种在推理时间具备信心意识的路由机制,以避免高信心和噪声以实现高效推理。ThinkRouter在模型信心低时将推理路由到离散标记空间,否则路由到潜在空间。在广泛的大型推理模型上对STEM推理和编码基准的广泛实验表明,ThinkRouter在准确率方面优于显式CoT、随机路由和潜在推理基线,平均在Pass@1上提高了19.70分,同时将生成长度减少多达15.55%。进一步的综合分析表明,ThinkRouter能够校准来自显式CoT和潜在推理的误差,并通过全局降低模型信心加速结束思考标记的生成。
英文摘要:
Recent work explores latent reasoning to improve reasoning efficiency by replacing explicit reasoning trajectories with continuous representations in a latent space, yet its effectiveness varies across settings. Analysis of model confidence dynamics under latent reasoning reveals that thinking trajectories ending in incorrect answers contain fewer low-confidence steps than those ending in correct answers. Meanwhile, we suggest that soft embeddings aggregated by multiple low-confidence thinking alternatives may introduce and propagate noise, leading to high confidence in unreliable reasoning trajectories. Motivated by these observations, ThinkRouter, an inference-time confidence-aware routing mechanism is proposed to avoid high confidence and noise for efficient reasoning. ThinkRouter routes thinking to the discrete token space when model confidence is low, and to the latent space otherwise. Extensive experiments on STEM reasoning and coding benchmarks across diverse large reasoning models demonstrate that ThinkRouter outperforms explicit CoT, random routing, and latent reasoning baselines in terms of accuracy, achieving an average improvement of 19.70 points in Pass@1, while reducing generation length by up to 15.55%. Further comprehensive analysis reveals that ThinkRouter can calibrate errors arising from explicit CoT and latent reasoning, and accelerates end-of-thinking token generation by globally lowering model confidence.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。