arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.06146cs.CLcs.AIcs.SE
收藏

使用语法掩码确保基于大语言模型的建模任务中的句法有效性

Using Grammar Masking to Ensure Syntactic Validity in LLM-based Modeling Tasks

Lukas Netz, Jan Reimer, Bernhard Rumpe

AI总结:

针对大语言模型建模时句法易出错、复杂文法下提示工程效果有限的问题,提出语法掩码方法,通过约束解码限制输出符合给定文法,实验证明其可显著提升多类LLM的建模正确率,降低对精细提示的依赖。

摘要:

我们提出并评估了一种名为语法掩码(grammar masking)的方法,用于引导大语言模型(LLM)生成符合给定上下文无关文法的句法正确模型。少样本学习或启动(priming)等提示工程方法可提升LLM生成正确句法的概率,但文法越复杂,这类方法就越耗时且效果越差。已有研究主要聚焦于语言模型训练或提示工程的应用。本研究提出一种方法,通过约束解码将输出限制在给定文法范围内,确保输出符合有效句法。我们使用多个基于MontiCore构建的领域特定语言(DSL),让多个LLM分别在有、无约束解码的条件下生成模型,并通过对应的解析器确认每个模型的句法正确性。实验表明,语法掩码可显著提升多个LLM的建模能力,既减少了对精细打磨提示的需求,又提高了生成正确模型的概率。

英文摘要:

We present and evaluate a method called grammar masking, which is used to guide large language models (LLMs) toward producing syntactically correct models for a given context-free grammar. Prompt engineering methods such as few-shot learning or priming can be used to improve the chances of an LLM producing correct syntax, but the more complex the grammar, the more time-consuming and less promising these methods become. Previous work is focused primarily on the usage of either language model training or prompt engineering. In this work, a method is presented that restricts the output to a given grammar using constrained decoding to ensure the output adheres to a valid syntax. We use several DSLs built with MontiCore and task multiple LLMs to produce models with and without constrained decoding. A corresponding parser is used to confirm the syntactic correctness of each model. We show that grammar masking can dramatically improve the modeling capabilities of several LLMs, reducing the need for well-refined prompting while increasing the chance of producing correct models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑