指令混合：通过混合提示对齐大语言模型
Mixture-of-Instructions: Aligning Large Language Models via Mixture Prompting
AI总结：
本文提出指令混合（MoI）技术，通过指令打包与多样化系统提示缓解LLM对系统提示的过拟合，并在Qwen-7B-chat上验证其在编程、数学和工具使用任务中的对齐效果提升。
摘要：
随着大语言模型（LLMs）的迅速普及，对此类模型在多个任务上的全面对齐已成为一个关键研究领域。现有的对齐方法主要针对单一任务，例如多轮对话、编程、数学问题求解和工具使用。尽管这些任务已有大量高质量数据，但其中大多数仅提供问题和答案，而未包含系统提示。通过对Qwen语言模型的详细分析，我们发现系统提示对LLM的训练和推理过程均有显著影响。我们将这一现象归因于对系统提示的过拟合。为解决该问题，我们提出了一种名为指令混合（Mixture-of-Instructions，MoI）的新技术，该技术采用指令打包策略并结合多样化的系统提示，以提升语言模型的对齐效率。我们还构建了包含七个基准数据集的多样化评测集合，以严格评估经MoI增强的语言模型的对齐效果。我们将该方法应用于开源Qwen-7B-chat模型，最终开发出Qwen-SFT-MoI。该增强模型在编程、数学和工具使用任务中展现出生成能力的显著提升。
英文摘要：
With the proliferation of large language models (LLMs), the comprehensive alignment of such models across multiple tasks has emerged as a critical area of research. Existing alignment methodologies primarily address single task, such as multi-turn dialogue, coding, mathematical problem-solving, and tool usage. Although there is a large amount of high-quality data available for those tasks, most of them provide only questions and answers without including the system prompt. Though a detailed analysis of the Qwen language model, we found that the system prompt has a significant impact on both training and inference processes of LLM. We attributes this phenomenon to overfitting to the system prompt. In address this issue, we introduce a novel technique termed Mixture-of-Instructions (MoI), which employs a strategy of instruction packing combined with diverse system prompts to boost the alignment efficiency of language models. We have also compiled a diverse set of seven benchmark datasets to rigorously evaluate the alignment efficacy of the MoI-enhanced language model. Our methodology was applied to the open-source Qwen-7B-chat model, culminating in the development of Qwen-SFT-MoI. This enhanced model demonstrates significant advancements in generative capabilities across coding, mathematics, and tool use tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
