揭示扩散大语言模型在可控生成中的潜力
Unveiling the Potential of Diffusion Large Language Model in Controllable Generation
AI总结：
针对自回归大模型在结构化输出上的不可靠性，提出基于扩散大模型的自适应模式脚手架框架，利用全局信息共享与反向推理能力，显著提升可控生成的稳定性与保真度。
摘要：
可控生成是自然语言处理中的一项基础任务，具有众多应用，为智能体通信中的函数调用提供了基础。然而，即使是当前最先进的自回归大语言模型（LLMs），在需要生成结构化输出时也表现出不可靠性。受当前新型基于扩散的大语言模型（dLLM）的启发，我们意识到架构差异，尤其是用于语言建模的全局信息共享机制，可能是解锁下一代可控生成的关键。为探索这一可能性，我们提出了自适应模式脚手架（$S^3$），一种新颖的框架，利用dLLM固有的反向推理能力和全局上下文感知，使其能够稳定地生成可靠的结构化输出（如JSON）。$S^3$直接在输出上下文中初始化一个模式模板作为dLLM的起始状态，提供了一种比复杂的提示优化更稳健、更通用的方法。实验表明，我们的方法在结构遵循性、内容保真度和忠实性方面显著释放了dLLM在可控生成中的潜力。这些结果为在可控生成任务中部署语言模型确立了新的视角和实用路径。
英文摘要：
Controllable generation is a fundamental task in NLP with many applications, providing a basis for function calling to agentic communication. However, even state-of-the-art autoregressive Large Language Models (LLMs) today exhibit unreliability when required to generate structured output. Inspired by the current new diffusion-based large language models (dLLM), we realize that the architectural difference, especially the global information-sharing mechanism for language modeling, may be the key to unlock next-level controllable generation. To explore the possibility, we propose Self-adaptive Schema Scaffolding ($S^3$), a novel framework that enables dLLM to stably generate reliable structured outputs (e.g., JSON) by utilizing its innate reverse reasoning capability and global context awareness. $S^3$ initiates a schematic template directly in the output context as a starting state for dLLM, offering a more robust and general method than intricate prompt optimization. Experiments demonstrate that our method substantially unlocks the dLLM's potential in controllable generation in terms of structure adherence, content fidelity, and faithfulness. These results establish new perspectives and practical pathways for deploying language models in controllable generation tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
