arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2305.15541cs.CLcs.AI
收藏

利用大型语言模型的力量进行自然语言到一阶逻辑的翻译

Harnessing the Power of Large Language Models for Natural Language to First-Order Logic Translation

Georgia Institute of Technology(佐治亚理工学院);Cisco(思科公司);Monash University(莫纳什大学)

Yuan Yang, Siheng Xiong, Ali Payani, Ehsan Shareghi, Faramarz Fekri

AI总结:

本文提出 LogicLLaMA,通过 LoRA 微调 LLaMA-7B 实现自然语言到一阶逻辑翻译,并结合 SFT+RLHF 框架赋予其纠正 GPT-3.5 输出、以低成本逼近 GPT-4 性能的能力。

摘要:

将自然语言句子翻译为一阶逻辑(NL-FOL 翻译)是自然语言处理和形式逻辑文献中长期存在的挑战。本文介绍了 LogicLLaMA,这是一个 LLaMA-7B 模型,使用 LoRA 在单个 GPU 上针对 NL-FOL 翻译进行了微调。LogicLLaMA 能够直接将自然语言翻译为 FOL 规则,其性能优于 GPT-3.5。LogicLLaMA 还具备纠正 GPT-3.5 预测的 FOL 规则的能力,并且能够以极低的成本达到与 GPT-4 相近的性能。这种纠正能力是通过一种新颖的监督微调(SFT)+ 基于人类反馈的强化学习(RLHF)框架实现的,该框架首先在合成扰动的 NL-FOL 对上训练,以鼓励思维链推理,然后使用 FOL 验证器作为奖励模型,在 GPT-3.5 的输出上通过 RLHF 进行微调。为了训练 LogicLLaMA,我们提出了 MALLS(大型语言模型生成的 NL-FOL 对),这是一个包含 34K 个高质量且多样化的句子级 NL-FOL 对的数据集,收集自 GPT-4。该数据集通过一个流水线创建,该流水线提示 GPT-4 生成对,并动态调整提示以确保收集到在不同复杂度级别上具有丰富多样上下文的对,同时验证生成的 FOL 规则的有效性。代码、权重和数据可在 https://github.com/gblackout/LogicLLaMA 获取。

英文摘要:

Translating natural language sentences to first-order logic (NL-FOL translation) is a longstanding challenge in the NLP and formal logic literature. This paper introduces LogicLLaMA, a LLaMA-7B model fine-tuned for NL-FOL translation using LoRA on a single GPU. LogicLLaMA is capable of directly translating natural language into FOL rules, which outperforms GPT-3.5. LogicLLaMA is also equipped to correct FOL rules predicted by GPT-3.5, and can achieve similar performance as GPT-4 with a fraction of the cost. This correction ability was achieved by a novel supervised fine-tuning (SFT) + reinforcement learning with human feedback (RLHF) framework, which initially trains on synthetically perturbed NL-FOL pairs to encourage chain-of-thought reasoning and then fine-tunes with RLHF on GPT-3.5 outputs using a FOL verifier as the reward model. To train LogicLLaMA, we present MALLS (large language $\textbf{M}$odel gener$\textbf{A}$ted N$\textbf{L}$-FO$\textbf{L}$ pair$\textbf{S}$), a dataset of 34K high-quality and diverse sentence-level NL-FOL pairs collected from GPT-4. The dataset was created by implementing a pipeline that prompts GPT-4 for pairs, and dynamically adjusts the prompts to ensure the collection of pairs with rich and diverse contexts at different levels of complexity, and verifies the validity of the generated FOL rules. Codes, weights, and data are available at $\href{https://github.com/gblackout/LogicLLaMA}{\small \text{https://github.com/gblackout/LogicLLaMA}}$.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑