可证明易处理的基于HMM的NFA约束语言生成

中文摘要 AI 辅助

约束生成旨在从受硬约束条件限制的语言模型（LMs）中进行采样。现有的针对非确定性有限自动机（NFA）约束的约束生成技术要么扭曲分布，要么牺牲效率。理论上，该任务可归结为计算被NFA接受的长度为n的序列的数量（#NFA），而精确的#NFA问题是#P完全的。最近的研究表明，#NFA具有完全多项式随机近似方案（FPRAS）。受此结果启发，我们提出了NFA-LM，一个在温和假设下具有理论保证的多项式时间NFA约束生成引擎。实验表明，NFA-LM能高效生成高质量输出，且具有理论上界有界的近似误差。

英文摘要

Constrained generation aims to sample from language models (LMs) conditioned on hard constraints. Existing constrained-generation techniques for nondeterministic finite automaton (NFA) constraints either distort the distribution or sacrifice efficiency. Theoretically, this task reduces to counting the length-$n$ sequences accepted by an NFA (#NFA), and the exact #NFA problem is #P-complete. Recent work has shown that #NFA admits a fully polynomial randomized approximation scheme (FPRAS). Inspired by this result, we propose NFA-LM, a polynomial-time engine for NFA-constrained generation with theoretical guarantees under mild assumptions. Experiments show that NFA-LM efficiently generates high-quality outputs with theoretically bounded approximation error.

发表机构

• University of Toronto（多伦多大学）

机构由 AI 辅助整理，请以论文原文为准。