arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.02823cs.CLcs.AIcs.LG
收藏

Conifer:提升大型语言模型的复杂约束指令遵循能力

Conifer: Improving Complex Constrained Instruction-Following Ability of Large Language Models

Alibaba Group(阿里巴巴集团)

Haoran Sun, Lixin Liu, Junjie Li, Fengyu Wang, Baohua Dong, Ran Lin, Ruohui Huang

AI总结:

本文提出 Conifer 指令微调数据集与渐进式学习方案,利用 GPT-4 精炼数据,显著提升 LLM 对复杂约束多层级指令的遵循能力。

摘要:

大型语言模型(LLM)遵循指令的能力对现实世界应用至关重要。尽管近期取得了进展,但多项研究强调,LLM 在面对具有挑战性的指令时仍会遇到困难,尤其是那些包含复杂约束的指令,这阻碍了它们在各类任务中的有效性。为应对这一挑战,我们提出 Conifer,这是一个新颖的指令微调数据集,旨在增强 LLM 遵循带有复杂约束的多层级指令的能力。我们利用 GPT-4,通过一系列由 LLM 驱动的精炼流程来构建数据集,以确保高质量。我们还提出一种渐进式学习方案,强调由易到难的推进过程,并从过程反馈中学习。使用 Conifer 训练的模型在指令遵循能力上表现出显著提升,尤其是在带有复杂约束的指令方面。在多个指令遵循基准测试中,我们的 7B 模型优于最先进的开源 7B 模型,甚至在某些指标上超过了规模大 10 倍的模型的性能。所有代码和 Conifer 数据集均可在 https://www.github.com/ConiferLM/Conifer 获取。

英文摘要:

The ability of large language models (LLMs) to follow instructions is crucial to real-world applications. Despite recent advances, several studies have highlighted that LLMs struggle when faced with challenging instructions, especially those that include complex constraints, hindering their effectiveness in various tasks. To address this challenge, we introduce Conifer, a novel instruction tuning dataset, designed to enhance LLMs to follow multi-level instructions with complex constraints. Utilizing GPT-4, we curate the dataset by a series of LLM-driven refinement processes to ensure high quality. We also propose a progressive learning scheme that emphasizes an easy-to-hard progression, and learning from process feedback. Models trained with Conifer exhibit remarkable improvements in instruction-following abilities, especially for instructions with complex constraints. On several instruction-following benchmarks, our 7B model outperforms the state-of-the-art open-source 7B models, even exceeds the performance of models 10 times larger on certain metrics. All the code and Conifer dataset are available at https://www.github.com/ConiferLM/Conifer.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑