UBTree：基于一元和二元模型的并行树状草稿生成用于投机解码

中文摘要 AI 辅助

投机解码通过在单次目标模型前向传播中验证多个草稿令牌来加速语言模型推理。最近的并行草稿生成器在前沿生产模型中取得了突破性性能，但随着目标分布熵的增加，由于草稿多样性不足，其有效性会下降。为了在不牺牲并行性的情况下克服这一瓶颈，我们引入了UBTree，一种并行草稿生成器，它将一元提议器与二元选择器耦合以构建草稿树。一元提议器使用标准交叉熵目标进行训练，为每个位置独立生成候选令牌，而轻量级二元选择器预测相邻候选对之间的转移分数。与提议器不同，选择器在高温度数据上使用重新归一化的KL目标进行训练。这种树原生训练将监督范围扩展到贪婪路径之外，鼓励合理的替代分支，从而提高树验证期间接受额外令牌的机会。在七个标准化基准上使用Qwen3-4B和Qwen3-8B，UBTree相对于自回归解码实现了平均$5.84$--$6.94\times$的加速，并在所有28次比较中优于DARTree。生产规模评估进一步证明了UBTree相对于DSpark等前沿基线的优势。

英文摘要

Speculative decoding accelerates language model inference by verifying multiple draft tokens in a single target-model pass. Recent parallel drafters have achieved breakthrough performance in frontier production models, but their effectiveness deteriorates as the entropy of target distributions increases due to insufficient draft diversity. To overcome this bottleneck without sacrificing parallelism, we introduce UBTree, a parallel drafter that couples a Unigram proposer with a Bigram selector to construct drafting Trees. The unigram proposer is trained with the standard cross-entropy objective to generate candidate tokens independently for each position, while a lightweight bigram selector predicts transition scores between adjacent candidate pairs. Unlike the proposer, the selector is trained with a renormalized KL objective on high-temperature data. This tree-native training broadens the supervision beyond the greedy path, encouraging plausible alternative branches that improve the chance of accepting additional tokens during tree verification. Across seven standardized benchmarks with Qwen3-4B and Qwen3-8B, UBTree achieves an average speedup of $5.84$--$6.94\times$ over autoregressive decoding and outperforms DARTree in all 28 comparisons. Production-scale evaluation further demonstrates UBTree's advantage over frontier baselines such as DSpark.

发表机构

• Inclusion AI
• University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
• Purdue University（普渡大学）

机构由 AI 辅助整理，请以论文原文为准。