带逻辑约束的自回归模型的伪语义损失
A Pseudo-Semantic Loss for Autoregressive Models with Logical Constraints
AI总结：
针对自回归分布下逻辑约束似然计算难的问题，提出基于伪似然的局部因子化近似损失，在数独、最短路径预测及大模型去毒任务上均取得性能提升。
摘要：
神经符号AI填补了纯符号学习方法与神经学习方法之间的鸿沟。这类研究通常需要最大化符号约束相对于神经网络输出分布的似然，而现有方法通常假设这类输出分布是完全因子化的，这限制了神经符号学习在更具表达能力的自回归分布（如transformer）上的适用性。在这类分布下，即便计算简单约束的似然也是#P难问题。我们并未尝试在整个输出分布上施加约束，而是提出在其随机局部近似上施加约束。更具体地说，我们在以模型样本为中心的基于伪似然的近似下优化约束的似然。我们的近似是因子化的，允许复用子问题的解，这是高效计算神经符号损失的核心原则。此外，它是似然的局部高保真近似，在模型样本周围表现出低熵和低KL散度。我们在被建模为自回归生成任务的数独和最短路径预测上评估了所提方法，观察到该方法大幅提升了基础模型预测逻辑一致输出的能力。我们还在大语言模型去毒任务上进行了评估，使用一个禁止特定有毒词列表的简单约束，就能引导模型输出避开有毒生成内容，与现有方法相比达到了SOTA去毒效果。
英文摘要：
Neuro-symbolic AI bridges the gap between purely symbolic and neural approaches to learning. This often requires maximizing the likelihood of a symbolic constraint w.r.t the neural network's output distribution. Such output distributions are typically assumed to be fully-factorized. This limits the applicability of neuro-symbolic learning to the more expressive autoregressive distributions, e.g., transformers. Under such distributions, computing the likelihood of even simple constraints is #P-hard. Instead of attempting to enforce the constraint on the entire output distribution, we propose to do so on a random, local approximation thereof. More precisely, we optimize the likelihood of the constraint under a pseudolikelihood-based approximation centered around a model sample. Our approximation is factorized, allowing the reuse of solutions to sub-problems, a main tenet for efficiently computing neuro-symbolic losses. Moreover, it is a local, high-fidelity approximation of the likelihood, exhibiting low entropy and KL-divergence around the model sample. We evaluate our approach on Sudoku and shortest-path prediction cast as autoregressive generation, and observe that we greatly improve upon the base model's ability to predict logically-consistent outputs. We also evaluate on the task of detoxifying large language models. Using a simple constraint disallowing a list of toxic words, we are able to steer the model's outputs away from toxic generations, achieving SoTA detoxification compared to previous approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
