连续扩散模型可以遵守形式语法
Continuous Diffusion Models Can Obey Formal Syntax
AI总结：
提出一种无需训练的引导方法，利用正则表达式约束连续扩散语言模型的生成过程，使其满足形式语法，并在JSON和自然语言基准上实现68-96%的约束满足率。
摘要：
扩散语言模型因其全局、非因果的生成过程而成为自回归模型的有前途的替代方案，但其连续潜在动态使得离散约束（例如，输出应为匹配给定模式的JSON文件）难以施加。我们提出了一种无需训练的引导方法，用于引导连续扩散语言模型满足用正则表达式表达的形式语法约束。我们的方法构建了一个解析分数，估计潜在状态解码为给定正则表达式接受的合法字符串的概率，并利用其梯度引导采样，无需训练辅助分类器。去噪过程以句法有效性为条件，针对基础模型进行优化。我们在PLAID扩散模型之上将我们的方法实现为Diffinity，并在180个正则表达式约束下对JSON和自然语言基准进行了评估。Diffinity实现了68-96%的约束满足率，同时相对于无约束采样仅产生很小的困惑度代价，在约束满足和输出质量方面均优于自回归约束解码。Diffinity已在github.com/large-loris-models/Diffinity开源。
英文摘要：
Diffusion language models offer a promising alternative to autoregressive models due to their global, non-causal generation process, but their continuous latent dynamics make discrete constraints -- e.g., the output should be a JSON file that matches a given schema -- difficult to impose. We introduce a training-free guidance method for steering continuous diffusion language models to satisfy formal syntactic constraints expressed using regular expressions. Our approach constructs an analytic score estimating the probability that a latent state decodes to a valid string accepted by a given regular expression, and uses its gradient to guide sampling, without training auxiliary classifiers. The denoising process targets the base model conditioned on syntactic validity. We implement our method in Diffinity on top of the PLAID diffusion model and evaluate it on 180 regular-expression constraints over JSON and natural-language benchmarks. Diffinity achieves 68-96\% constraint satisfaction while incurring only a small perplexity cost relative to unconstrained sampling, outperforming autoregressive constrained decoding in both constraint satisfaction and output quality. Diffinity is open-sourced at github.com/large-loris-models/Diffinity.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
