分而治之：通过自适应并行解码加速扩散式大语言模型
Divide and Conquer: Accelerating Diffusion-Based Large Language Models via Adaptive Parallel Decoding
AI总结：
DiCo通过分而治之的自适应并行解码方法，提升扩散式大语言模型的推理速度并保持生成质量。
摘要：
基于扩散的大型语言模型（dLLMs）在各种推理任务中表现出有希望的性能，已建立自身作为自回归大型语言模型（LLMs）的替代方案。与自回归LLMs不同，后者每一步生成一个token基于所有之前的token，dLLMs理论上允许在每个解码步骤中并行生成多个token。然而，最近的dLLMs在实践中仍然倾向于每步生成一个token，因为直接解码多个被遮蔽的token往往会导致生成质量下降和稳定性降低。这揭示了dLLMs理论并行性和实际性能之间存在显著差距。为了弥合这一差距，我们引入了一种自适应并行解码方法，即DiCo，其特征是三阶段的分而治之范式，以释放dLLMs的内在并行性。在Divide阶段，DiCo首先探索输入被遮蔽的序列，并识别被遮蔽的token作为种子token，然后将其扩展以构建一组局部集群。在Conquer阶段，DiCo在Divide阶段构建的不同局部集群上执行并行解码。分而治之的过程在Divide和Conquer阶段之间反复交替，直到收敛。在Finalize阶段，DiCo使用有效的细粒度复合解码方案解码剩余的几个被遮蔽的token以完成生成。广泛的实验表明，DiCo可以在保持竞争性生成质量的同时实现显著的推理加速。
英文摘要：
Diffusion-based large language models (dLLMs) have shown promising performance across various reasoning tasks, establishing themselves as an alternative to autoregressive large language models (LLMs). Unlike autoregressive LLMs that generate one token per step based on all previous tokens, dLLMs theoretically enable parallel generation of multiple tokens at each decoding step. However, recent dLLMs still favor one-token-per-step generation in practice, as directly decoding multiple masked tokens often leads to degraded generation quality and stability. This reveals a substantial gap between the theoretical parallelism and practical performance of dLLMs. To bridge this gap, we introduce an adaptive parallel decoding approach, namely DiCo, which features a three-phase divide-and-conquer paradigm to unleash the inherent parallelism of dLLMs. During the Divide phase, DiCo first explores the input masked sequence and identifies masked tokens as seed tokens, which are then expanded to construct a set of local clusters. During the Conquer phase, DiCo performs parallel decoding across different local clusters constructed in the Divide phase. The divide-and-conquer process repeatedly alternates between the Divide and Conquer phases until convergence. During the Finalize phase, DiCo decodes the remaining few masked tokens using an effective fine-grained compound decoding scheme to finalize the generation. Extensive experiments demonstrate that DiCo can achieve significant inference speedups while maintaining competitive generation quality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
