PSD: 推动扩散大语言模型的帕累托前沿：通过并行推测解码
PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding
AI总结：
本文提出PSD框架，通过并行推测解码提升推理效率与生成质量，在推理效率和生成质量之间取得良好平衡，达到每前向传递5.5倍的token处理速度。
摘要：
扩散大语言模型（dLLMs）通过迭代去噪掩码标记序列生成文本。尽管dLLMs可以在每个步骤内并行预测所有掩码位置，但大量的去噪迭代仍使推理成本高昂。此成本可通过每步解掩多个标记进行空间优化，或通过将多个去噪步骤合并为一次验证调用进行时间优化。我们提出并行推测解码（PSD），一种无需训练的框架，同时提升推理效率和生成质量。利用单次前向传递的置信度分数，PSD通过可配置的自适应解掩策略选择解掩位置，并构建多深度的推测草案而无需额外模型调用。最终的批量验证步骤应用分层接受机制，保留与更新预测一致的最深草案。在三个dLLMs上进行的实验表明，PSD在推理效率和生成质量之间取得了良好的权衡，达到每前向传递5.5倍的token处理速度，其准确性与贪婪解码相当。
英文摘要：
Diffusion large language models (dLLMs) generate text by iteratively denoising masked token sequences. Although dLLMs can predict all masked positions in parallel within each step, the large number of denoising iterations still makes inference expensive. This cost can be reduced spatially by unmasking multiple tokens per step, or temporally by collapsing multiple denoising steps into one verification call. We propose Parallel Speculative Decoding (PSD), a training-free framework that jointly improves inference along both axes. Using the confidence scores from a single forward pass, PSD selects positions to unmask via a configurable, adaptive unmasking policy and constructs multi-depth speculative drafts without extra model calls. A final batched verification pass then applies hierarchical acceptance, keeping the deepest draft that remains consistent with the updated predictions. Experiments on three dLLMs across reasoning and code generation tasks show that PSD achieves favorable trade-offs between inference efficiency and generation quality, reaching up to $5.5\times$ tokens per forward pass with accuracy comparable to greedy decoding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
