扩展谜题以用于专家混合推理模型的推理,应用于GPT-OSS加速
Extending Puzzle for Mixture-of-Experts Reasoning Models with Application to GPT-OSS Acceleration
AI总结:
本文通过扩展Puzzle框架优化GPT-OSS模型,结合MoE剪枝、注意力替换、量化和强化学习提升推理效率,实现部署优化并提高请求级效率。
摘要:
面向推理的LLM通过生成更长的推理轨迹提高回答质量,但额外的token显著增加服务成本,推动了推理优化。我们扩展并应用Puzzle,一种后训练神经网络架构搜索(NAS)框架,到gpt-oss-120B以生成gpt-oss-puzzle-88B,一种部署优化的衍生模型。我们的方法结合异构MoE专家剪枝、选择性替换全上下文注意力为窗口注意力、FP8 KV缓存量化与校准尺度,以及后训练强化学习以恢复准确性,同时保持低生成长度。在8XH100节点上,我们实现了长上下文和短上下文设置下的1.63X和1.22X吞吐量提升。gpt-oss-puzzle-88B还在单个NVIDIA H100 GPU上实现了2.82X的吞吐量提升。然而,由于token数量可能随着推理努力和模型变体而变化,每token吞吐量(tok/s)和延迟(ms/token)不必然导致端到端加速:2X吞吐量提升如果轨迹增长2X则会被抵消。相反,吞吐量提升可以用于更多推理token以提高准确性;因此,我们倡导请求级效率度量,将吞吐量归一化为生成的token,并在推理努力中追踪准确性-速度前沿。我们证明gpt-oss-puzzle-88B在前沿上优于gpt-oss-120B,最高达到1.29X的请求级效率提升。在各种基准测试中,gpt-oss-puzzle-88B在推理努力中匹配或略微超过父模型的套件平均准确性,保留率范围从100.8%(高)到108.2%(低),表明后训练架构搜索可以显著降低推理成本而不牺牲质量。
英文摘要:
Reasoning-focused LLMs improve answer quality by generating longer reasoning traces, but the additional tokens dramatically increase serving cost, motivating inference optimization. We extend and apply Puzzle, a post-training neural architecture search (NAS) framework, to gpt-oss-120B to produce gpt-oss-puzzle-88B, a deployment-optimized derivative. Our approach combines heterogeneous MoE expert pruning, selective replacement of full-context attention with window attention, FP8 KV-cache quantization with calibrated scales, and post-training reinforcement learning to recover accuracy, while maintaining low generation length. In terms of per-token speeds, on an 8XH100 node we achieve 1.63X and 1.22X throughput speedups in long-context and short-context settings, respectively. gpt-oss-puzzle-88B also delivers throughput speedups of 2.82X on a single NVIDIA H100 GPU. However, because token counts can change with reasoning effort and model variants, per-token throughput (tok/s) and latency (ms/token) do not necessarily lead to end-to-end speedups: a 2X throughput gain is erased if traces grow 2X. Conversely, throughput gains can be spent on more reasoning tokens to improve accuracy; we therefore advocate request-level efficiency metrics that normalize throughput by tokens generated and trace an accuracy--speed frontier across reasoning efforts. We show that gpt-oss-puzzle-88B improves over gpt-oss-120B along the entire frontier, delivering up to 1.29X higher request-level efficiency. Across various benchmarks, gpt-oss-puzzle-88B matches or slightly exceeds the parent on suite-average accuracy across reasoning efforts, with retention ranging from 100.8% (high) to 108.2% (low), showing that post-training architecture search can substantially reduce inference costs without sacrificing quality.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。