arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.10273stat.MLcs.LG
收藏

Power-SMC:低延迟序列级功率采样用于训练自由LLM推理

Power-SMC: Low-Latency Sequence-Level Power Sampling for Training-Free LLM Reasoning

University of Southern California(南加州大学);Intel Labs(英特尔实验室)

Seyedarmin Azizi, Erfan Baghaei Potraghloo, Minoo Ahmadi, Souvik Kundu, Massoud Pedram

AI总结:

本文提出Power-SMC,一种低延迟的序列级功率采样方法,用于训练自由的大语言模型推理,通过并行粒子集和重要权重修正提升推理效率。

摘要:

许多近期大语言模型推理能力的提升可以解释为分布锐化:将生成偏向于由预训练模型支持的高似然轨迹,而不是修改其权重。自然形式化是序列级功率分布π_α(y|x)∝p_θ(y|x)^α(α>1),该分布将质量集中在完整序列而非调整token级温度。先前工作表明,从该分布中使用Metropolis-Hastings(MH)采样可恢复强大的推理性能,但导致数量级的推理延迟增加。本文引入Power-SMC,一种无训练的顺序蒙特卡洛方案,旨在达到相同的目标同时保持接近标准解码延迟。Power-SMC并行推进小粒子集,逐token修正重要权重,并在必要时重新采样,所有操作均在单个GPU友好的批量解码中完成。我们证明温度τ=1/α是唯一前缀仅提案,最小化增量权重方差,通过前缀条件化Rényi熵解释残差不稳定性,并引入指数桥接计划,提高粒子稳定性而不改变目标。在MATH500上,Power-SMC匹配或超过MH功率采样,同时将延迟从16-28倍降低到1.4-3.3倍。代码可在https://github.com/ArminAzizi98/Power-SMC获取。

英文摘要:

Reasoning ability in large language models is often attributed to \emph{distribution sharpening}: concentrating output probability on high-likelihood sequences. Recent works show that this sharpening effect can be obtained at inference time, without modifying model parameters, and can elicit strong reasoning performance. A natural formalization is the \emph{sequence-level power distribution}, which is proportional to the model's probability raised to an exponent $α>1$. Prior work leveraged Metropolis--Hastings (MH) sampling to draw samples from this distribution and achieves strong results, however, at order-of-magnitude inference slowdowns. We introduce \textbf{Power-SMC}, a \textit{`training-free'} sampling method that targets the same power distribution yielding close to standard decoding latency. Power-SMC maintains multiple candidate sequences in parallel. Each candidate sequence is assigned a score, namely the \emph{importance weight}, that measures how well it matches the power distribution. It then periodically prunes low-scoring candidate sequences in favor of high-scoring ones. We further provide a theoretical justification for the design choices in Power-SMC. Among all next-token sampling strategies that do not rely on future tokens, we prove that sampling temperature $τ{=}1/α$ uniquely eliminates per-step weight variance. Finally, we characterize the remaining source of weight instability and introduce a gradual sharpening schedule to reduce weight collapse, while targeting the same power distribution. Extensive evaluations on MATH500, GSM8K, GPQA, and HumanEval show that Power-SMC matches or exceeds MH sampling in accuracy, preserves output diversity unlike RL-finetuned models, while \textbf{accelerating inference speed by up to} {$\mathbf{17.6}\times$}. The code is available at https://github.com/ArminAzizi98/Power-SMC.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑