LLaDA2.1: 通过令牌编辑加速文本扩散
LLaDA2.1: Speeding Up Text Diffusion via Token Editing
AI总结：
LLaDA2.1通过结合令牌编辑与掩码到令牌方案，实现解码速度与生成质量的平衡，展示了在大规模扩散模型上的性能提升。
摘要：
尽管LLaDA2.0展示了100B级块扩散模型的扩展潜力及其固有的并行化能力，但解码速度与生成质量之间的微妙平衡仍然是一个难以捉摸的前沿。今天，我们推出LLaDA2.1，这是一种变革性的方法，旨在超越这一权衡。通过无缝地将令牌到令牌（T2T）编辑融入传统的掩码到令牌（M2T）方案中，我们引入了一种联合、可配置的阈值解码方案。这种结构创新催生了两种不同的身份：速度模式（S模式），它大胆地降低M2T阈值以绕过传统限制，同时依靠T2T来细化输出；以及质量模式（Q模式），它倾向于保守的阈值以确保优越的基准表现，同时保持可控的效率下降。进一步推动这一进化，依托于一个广阔的上下文窗口，我们实现了首个大规模强化学习（RL）框架，专门针对dLLMs进行优化，通过专门的技术进行稳定的梯度估计。这种对齐不仅提高了推理精度，还提高了遵循指令的保真度，弥合了扩散动态与复杂人类意图之间的鸿沟。我们通过发布LLaDA2.1-Mini（16B）和LLaDA2.1-Flash（100B）来结束这项工作。在33个严格基准测试中，LLaDA2.1在任务性能和闪电般的解码速度上表现强劲。尽管其100B的规模，但在编码任务上，它在HumanEval+上达到了惊人的892 TPS，在BigCodeBench上达到801 TPS，在LiveCodeBench上达到663 TPS。
英文摘要：
While LLaDA2.0 showcased the scaling potential of 100B-level block-diffusion models and their inherent parallelization, the delicate equilibrium between decoding speed and generation quality has remained an elusive frontier. Today, we unveil LLaDA2.1, a paradigm shift designed to transcend this trade-off. By seamlessly weaving Token-to-Token (T2T) editing into the conventional Mask-to-Token (M2T) scheme, we introduce a joint, configurable threshold-decoding scheme. This structural innovation gives rise to two distinct personas: the Speedy Mode (S Mode), which audaciously lowers the M2T threshold to bypass traditional constraints while relying on T2T to refine the output; and the Quality Mode (Q Mode), which leans into conservative thresholds to secure superior benchmark performances with manageable efficiency degrade. Furthering this evolution, underpinned by an expansive context window, we implement the first large-scale Reinforcement Learning (RL) framework specifically tailored for dLLMs, anchored by specialized techniques for stable gradient estimation. This alignment not only sharpens reasoning precision but also elevates instruction-following fidelity, bridging the chasm between diffusion dynamics and complex human intent. We culminate this work by releasing LLaDA2.1-Mini (16B) and LLaDA2.1-Flash (100B). Across 33 rigorous benchmarks, LLaDA2.1 delivers strong task performance and lightning-fast decoding speed. Despite its 100B volume, on coding tasks it attains an astounding 892 TPS on HumanEval+, 801 TPS on BigCodeBench, and 663 TPS on LiveCodeBench.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
