面向大语言模型的自适应逻辑控制
Adaptable Logical Control for Large Language Models
AI总结:
针对大语言模型推理阶段生成控制难的问题,提出结合LLM与隐马尔可夫模型的自适应逻辑控制框架Ctrl-G,在交互式文本编辑等任务上表现优于主流模型,还可拓展至推理辅助等场景。
摘要:
尽管大语言模型(LLMs)在遵循人类指令完成各类任务方面已取得成功,但在推理阶段控制模型生成仍是一项长期存在的挑战。本文提出了Ctrl-G,这是一个自适应框架,可实现对LLM生成的易处理且灵活的控制,使其可靠地遵循逻辑约束。Ctrl-G将任意可投产的LLM与隐马尔可夫模型(Hidden Markov Model)相结合,使LLM输出能够符合以确定性有限自动机表示的逻辑约束。我们的研究表明,将Ctrl-G应用于TULU2-7B模型时,在交互式文本编辑任务上的表现优于GPT3.5和GPT4:具体而言,在遵循逻辑约束生成文本插入/续写的任务中,Ctrl-G在人工评估中的满足率比GPT4高出30%以上。当应用于中等规模语言模型(例如GPT2-large)时,Ctrl-G在标准基准测试中的约束生成效果也大幅优于同类模型。此外,作为概念验证研究,我们在Grade School Math基准上测试了Ctrl-G以辅助LLM推理,预示了Ctrl-G及其他约束生成方法在传统语言生成任务之外的应用前景。
英文摘要:
Despite the success of Large Language Models (LLMs) on various tasks following human instructions, controlling model generation at inference time poses a persistent challenge. In this paper, we introduce Ctrl-G, an adaptable framework that facilitates tractable and flexible control of LLM generation to reliably follow logical constraints. Ctrl-G combines any production-ready LLM with a Hidden Markov Model, enabling LLM outputs to adhere to logical constraints represented as deterministic finite automata. We show that Ctrl-G, when applied to a TULU2-7B model, outperforms GPT3.5 and GPT4 on the task of interactive text editing: specifically, for the task of generating text insertions/continuations following logical constraints, Ctrl-G achieves over 30% higher satisfaction rate in human evaluation compared to GPT4. When applied to medium-size language models (e.g., GPT2-large), Ctrl-G also beats its counterparts for constrained generation by large margins on standard benchmarks. Additionally, as a proof-of-concept study, we experiment Ctrl-G on the Grade School Math benchmark to assist LLM reasoning, foreshadowing the application of Ctrl-G, as well as other constrained generation approaches, beyond traditional language generation tasks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。