arXiv 2505.119222025-05-17cs.CL

利用混合上下文微调增强大型语言模型的复杂指令遵循能力

Enhancing Complex Instruction Following for Large Language Models with Mixture-of-Contexts Fine-tuning

School of Artificial Intelligence, Beijing University of Posts and Telecommunications（人工智能学院，北京邮电大学）；LI Auto Inc.（LI汽车公司）

Yuheng Lu, ZiMeng Bai, Caixia Yuan, Huixing Jiang, Xiaojie Wang

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出 MISO 多输入单输出架构与混合上下文微调范式，通过并行子上下文增强 SFT，显著提升 LLM 对复杂多约束指令的遵循能力及训练效率。

摘要：

大型语言模型（LLM）在处理自然语言任务方面表现出卓越能力；然而，它们可能难以稳定地遵循复杂指令，包括那些涉及多重约束的指令。使用监督微调（SFT）对 LLM 进行后训练是提升其指令遵循能力的标准方法。在解决复杂指令遵循问题时，现有工作主要聚焦于数据驱动方法，即为 SFT 合成复杂的指令—输出对。然而，对关键子上下文关注不足可能会降低 SFT 的有效性。在本工作中，我们提出将顺序结构化的输入指令转换为多个包含子上下文的并行指令。为支持处理这种多输入，我们提出 MISO（Multi-Input Single-Output，多输入单输出），它是对当前占主导地位的基于 decoder-only transformer 的 LLM 的扩展。MISO 引入了混合上下文范式，联合考虑整体指令—输出对齐以及各个子上下文的影响，以增强 SFT 的有效性。我们将 MISO 微调应用于复杂指令遵循数据集，并使用标准 LLM 推理对其进行评估。实证结果表明，MISO 作为一种面向 LLM 的微调方法具有优越性，既体现在复杂指令遵循场景中的有效性，也体现在其训练效率潜力方面。

英文摘要：

Large language models (LLMs) exhibit remarkable capabilities in handling natural language tasks; however, they may struggle to consistently follow complex instructions including those involve multiple constraints. Post-training LLMs using supervised fine-tuning (SFT) is a standard approach to improve their ability to follow instructions. In addressing complex instruction following, existing efforts primarily focus on data-driven methods that synthesize complex instruction-output pairs for SFT. However, insufficient attention allocated to crucial sub-contexts may reduce the effectiveness of SFT. In this work, we propose transforming sequentially structured input instruction into multiple parallel instructions containing subcontexts. To support processing this multi-input, we propose MISO (Multi-Input Single-Output), an extension to currently dominant decoder-only transformer-based LLMs. MISO introduces a mixture-of-contexts paradigm that jointly considers the overall instruction-output alignment and the influence of individual sub-contexts to enhance SFT effectiveness. We apply MISO fine-tuning to complex instructionfollowing datasets and evaluate it with standard LLM inference. Empirical results demonstrate the superiority of MISO as a fine-tuning method for LLMs, both in terms of effectiveness in complex instruction-following scenarios and its potential for training efficiency.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。