CoachLM：自动指令修订提升 LLM 指令微调中的数据质量
CoachLM: Automatic Instruction Revisions Improve the Data Quality in LLM Instruction Tuning
AI总结：
本文提出 CoachLM，通过人类专家修订样本训练的自动指令修订方法，显著提升 LLM 指令微调数据质量，高质量样本比例从 17.7% 增至 78.9%，指令遵循能力平均提升 29.9%，并已在华为系统中部署应用。
摘要：
指令微调对于使语言学习模型（LLM）能够响应人类指令至关重要。用于微调的指令对的质量极大地影响 LLM 的性能。然而，手动创建高质量的指令数据集成本高昂，导致采用 LLM 自动生成指令对作为流行的替代方案。为了确保 LLM 生成的指令数据集的高质量，已经提出了几种方法。尽管如此，现有方法要么通过过滤掉大部分样本来损害数据集的完整性，要么不适用于工业应用。在本文中，我们不丢弃低质量样本，而是提出 CoachLM，一种通过对数据集中的样本进行自动修订来提升指令数据集质量的新颖方法。CoachLM 从由人类专家修订的样本中训练，并显著增加了数据集中高质量样本的比例，从 17.7% 提升至 78.9%。CoachLM 的有效性在各种真实世界指令测试集上得到了进一步评估。结果表明，CoachLM 将指令微调后的 LLM 的指令遵循能力平均提升了 29.9%，甚至超越了参数数量接近两倍的更大 LLM。此外，CoachLM 已成功部署在华为的 LLM 数据管理系统中，在清理 40k 真实世界指令对时实现了高达 20% 的效率提升。我们发布了 CoachLM 的各种资源，包括训练数据、代码和测试集（https://github.com/lunyiliu/CoachLM）。
英文摘要：
Instruction tuning is crucial for enabling Language Learning Models (LLMs) in responding to human instructions. The quality of instruction pairs used for tuning greatly affects the performance of LLMs. However, the manual creation of high-quality instruction datasets is costly, leading to the adoption of automatic generation of instruction pairs by LLMs as a popular alternative. To ensure the high quality of LLM-generated instruction datasets, several approaches have been proposed. Nevertheless, existing methods either compromise dataset integrity by filtering a large proportion of samples, or are unsuitable for industrial applications. In this paper, instead of discarding low-quality samples, we propose CoachLM, a novel approach to enhance the quality of instruction datasets through automatic revisions on samples in the dataset. CoachLM is trained from the samples revised by human experts and significantly increases the proportion of high-quality samples in the dataset from 17.7% to 78.9%. The effectiveness of CoachLM is further assessed on various real-world instruction test sets. The results show that CoachLM improves the instruction-following capabilities of the instruction-tuned LLM by an average of 29.9%, which even surpasses larger LLMs with nearly twice the number of parameters. Furthermore, CoachLM is successfully deployed in a data management system for LLMs at Huawei, resulting in an efficiency improvement of up to 20% in the cleaning of 40k real-world instruction pairs. We release various assets of CoachLM, including the training data, code and test set (https://github.com/lunyiliu/CoachLM).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
