套利:通过优势感知投机实现高效推理
Arbitrage: Efficient Reasoning via Advantage-Aware Speculation
AI总结:
Arbitrage通过动态路由机制提升步骤级投机生成的效率和准确性,减少推理延迟。
摘要:
现代大语言模型通过长链思维实现了令人印象深刻的推理能力,但推理过程中会带来显著的计算成本,这促使了改进性能-成本比的技术。其中,投机解码通过使用一个快速但不准确的草案模型自回归地提出token,然后由更强大的目标模型并行验证。然而,由于语义等价步骤中token不匹配导致的不必要的拒绝,传统token级投机解码在推理任务中表现不佳。尽管近期的工作转向了步骤级语义验证,通过接受或拒绝整个推理步骤来提高效率,现有步骤级方法仍然会重新生成许多被拒绝的步骤,改进有限,浪费了宝贵的计算资源。为了解决这一挑战,我们提出Arbitrage,一种新的步骤级投机生成框架,根据草案和目标模型之间的相对优势动态路由生成。而不是应用一个固定的接受阈值,Arbitrage使用一个轻量级的路由器来预测目标模型何时会产生更有意义的步骤。这种路由近似了一个理想的套利Oracle,总是选择高质量的步骤,实现了接近最优的效率-准确性权衡。在多个数学推理基准测试中,Arbitrage始终优于先前的步骤级投机解码基线,在匹配精度下将推理延迟减少了多达约2倍。
英文摘要:
Modern Large Language Models achieve impressive reasoning capabilities with long Chain of Thoughts, but they incur substantial computational cost during inference, and this motivates techniques to improve the performance-cost ratio. Among these techniques, Speculative Decoding accelerates inference by employing a fast but inaccurate draft model to autoregressively propose tokens, which are then verified in parallel by a more capable target model. However, due to unnecessary rejections caused by token mismatches in semantically equivalent steps, traditional token-level Speculative Decoding struggles in reasoning tasks. Although recent works have shifted to step-level semantic verification, which improve efficiency by accepting or rejecting entire reasoning steps, existing step-level methods still regenerate many rejected steps with little improvement, wasting valuable target compute. To address this challenge, we propose Arbitrage, a novel step-level speculative generation framework that routes generation dynamically based on the relative advantage between draft and target models. Instead of applying a fixed acceptance threshold, Arbitrage uses a lightweight router trained to predict when the target model is likely to produce a meaningfully better step. This routing approximates an ideal Arbitrage Oracle that always chooses the higher-quality step, achieving near-optimal efficiency-accuracy trade-offs. Across multiple mathematical reasoning benchmarks, Arbitrage consistently surpasses prior step-level Speculative Decoding baselines, reducing inference latency by up to $\sim2\times$ at matched accuracy.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。