学习与强制上下文敏感控制用于大语言模型
Learning and Enforcing Context-Sensitive Control for LLMs
AI总结：
本文提出一种自动学习上下文敏感约束的框架，通过两阶段过程提升LLM生成的有效性，使小型模型也能完美遵守约束，优于其他方法。
摘要：
通过上下文敏感约束控制大语言模型（LLMs）的输出已成为克服上下文无关文法（CFGs）在保证生成有效性方面局限的一种有前途的方法。然而，此类约束通常需要手动指定——这成为显著障碍，需要专门知识。我们引入了一个框架，通过两阶段过程自动从LLM交互中学习上下文敏感约束：语法探索以收集多样化的输出用于约束学习，随后是约束利用以在生成过程中强制这些学习规则。实验表明，我们的方法使即使是小型LLM（1B参数）也能学习并生成，完美遵守约束，优于更大的模型和最先进的推理模型。这项工作代表了首次将上下文敏感文法学习与LLM生成相结合，消除了手动指定的同时保持生成的有效性。
英文摘要：
Controlling the output of Large Language Models (LLMs) through context-sensitive constraints has emerged as a promising approach to overcome the limitations of Context-Free Grammars (CFGs) in guaranteeing generation validity. However, such constraints typically require manual specification -- a significant barrier demanding specialized expertise. We introduce a framework that automatically learns context-sensitive constraints from LLM interactions through a two-phase process: syntactic exploration to gather diverse outputs for constraint learning, followed by constraint exploitation to enforce these learned rules during generation. Experiments demonstrate that our method enables even small LLMs (1B parameters) to learn and generate with perfect constraint adherence, outperforming larger counterparts and state-of-the-art reasoning models. This work represents the first integration of context-sensitive grammar learning with LLM generation, eliminating manual specification while maintaining generation validity.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
