发表机构
School of Computer Science, Shanghai Jiao Tong University; Zhongguancun Academy（上海交通大学计算机科学与工程学院; 中关村科学城）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究扩散大语言模型中多令牌提交问题，提出CoCommit方法，通过标记门控协调步骤延迟提交，重新应用骨干网络最后n层使标记位置协调，近似联合模式解码，提高推理和精确答案任务准确率。
AI 中文摘要
扩散大语言模型（dLLMs）在每个去噪步骤中通过独立于共享上下文解码每个选定位置来提交多个令牌；当这些位置相关时，条件总相关性会捕获由此产生的因式分解误差，基于置信度的选择无法仅从边缘分布中观察到。我们提出了CoCommit，一种标记门控协调步骤，它会短暂延迟提交：在通常的束选择之后，一个学习到的标记宣布提交集，并且骨干网络的最后n层会重新应用，以便标记位置在贪婪argmax写入令牌之前进行协调——近似联合模式解码。该方法通过一次额外的部分前向传递重用现有权重，无需辅助模型。在使用LoRA适配器的LLaDA2.1-mini和匹配的贪婪推理上，联合提交提高了我们评估的所有六个基准的准确率，在推理和精确答案任务上的提升最大。
英文摘要
Diffusion language models (dLLMs) commit multiple tokens per denoising step by decoding each selected position independently from a shared context. When these positions are dependent, this factorization introduces an error captured by conditional total correlation, which confidence-based selection cannot infer from marginal probabilities alone. We propose CoCommit, a marker-gated coordination pass that delays commitment. After the usual bundle selection, a learned marker identifies the commit set, and the backbone's last n layers are re-applied to coordinate the marked positions before greedy argmax writes the tokens. This approximates joint-mode decoding while reusing existing weights, requiring only one partial forward pass and no auxiliary model. On LLaDA 2.1 with LoRA adapters and greedy inference, joint commitment improves five of the seven evaluated benchmarks over the released factorized decoder. The largest gains occur on code and reasoning tasks, while the remaining tasks are near parity.
