RelayGen: 生成内代际模型切换以实现高效推理
RelayGen: Intra-Generation Model Switching for Efficient Reasoning
AI总结:
RelayGen通过段级运行时模型切换,在不增加训练成本的情况下,提升推理效率并减少延迟,同时保持高精度。
摘要:
大型推理模型(LRMs)通过生成长且多步骤的推理轨迹,在复杂推理任务上取得优异表现,但推理时的扩展却带来了显著的部署成本。一个关键挑战是单个输出中生成难度存在差异,而现有以效率为导向的方法要么忽略这种生成内差异,要么依赖于监督的token级路由,具有较高的系统复杂性。我们提出了RelayGen,一种无需训练的段级运行时模型切换框架,利用长形式推理中的难度变化。通过离线分析生成不确定性使用token概率边际,我们证明粗粒度的段级控制足以捕捉推理轨迹中的难度转换。RelayGen识别出模型特定的切换提示,信号转换到低难度段,并动态将这些段的延续委托给较小的模型,同时在大模型上保持高难度推理。在多个推理基准测试中,RelayGen显著减少了推理延迟,同时保持了大模型的大部分准确性。当与推测解码结合时,RelayGen实现了高达2.2倍的端到端加速,精度下降不到2%,无需额外训练或学习路由组件。
英文摘要:
Large reasoning models (LRMs) achieve strong performance on complex reasoning tasks by generating long, multi-step reasoning trajectories, but inference-time scaling incurs substantial deployment cost. A key challenge is that generation difficulty varies within a single output, whereas existing efficiency-oriented approaches either ignore this intra-generation variation or rely on supervised token-level routing with high system complexity. We present \textbf{RelayGen}, a training-free, segment-level runtime model switching framework that exploits difficulty variation in long-form reasoning. Through offline analysis of generation uncertainty using token probability margins, we show that coarse-grained segment-level control is sufficient to capture difficulty transitions within a reasoning trajectory. RelayGen identifies model-specific switch cues that signal transitions to lower-difficulty segments and dynamically delegates their continuation to a smaller model, while preserving high-difficulty reasoning on the large model. Across multiple reasoning benchmarks, RelayGen substantially reduces inference latency while preserving most of the accuracy of large models. When combined with speculative decoding, RelayGen achieves up to 2.2$\times$ end-to-end speedup with less than 2\% accuracy degradation, without requiring additional training or learned routing components.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。