从数量到质量：通过自引导数据选择提升大语言模型指令微调性能
From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning
AI总结：
本文提出基于指令遵循难度（IFD）指标的自引导数据选择方法，使大语言模型能自主挑选高质量樱桃样本，仅用10%数据即可提升指令微调效率与性能。
摘要：
在大语言模型（LLMs）领域，指令数据的质量与数量之间的平衡是一个焦点问题。基于这一认识，我们提出了一种自引导方法，使大语言模型能够自主地从开源数据集中辨别并挑选“樱桃样本”，从而在指令微调大语言模型时有效减少人工筛选和潜在成本。我们的核心创新是指令遵循难度（Instruction-Following Difficulty，IFD）指标，该指标成为识别模型期望响应与其内在生成能力之间差异的关键度量。通过应用IFD，可以精准定位樱桃样本，从而显著提升模型训练效率。在Alpaca和WizardLM等数据集上的实证验证支撑了我们的发现；仅使用原始数据输入的$10\%$，我们的策略就展现出了更优的结果。这种自引导樱桃样本挑选与IFD指标的结合，标志着大语言模型指令微调的一次变革性飞跃，有望带来高效且资源节约的进步。代码、数据和模型已公开：https://github.com/tianyi-lab/Cherry_LLM
英文摘要：
In the realm of Large Language Models (LLMs), the balance between instruction data quality and quantity is a focal point. Recognizing this, we introduce a self-guided methodology for LLMs to autonomously discern and select cherry samples from open-source datasets, effectively minimizing manual curation and potential cost for instruction tuning an LLM. Our key innovation, the Instruction-Following Difficulty (IFD) metric, emerges as a pivotal metric to identify discrepancies between a model's expected responses and its intrinsic generation capability. Through the application of IFD, cherry samples can be pinpointed, leading to a marked uptick in model training efficiency. Empirical validations on datasets like Alpaca and WizardLM underpin our findings; with a mere $10\%$ of original data input, our strategy showcases improved results. This synthesis of self-guided cherry-picking and the IFD metric signifies a transformative leap in the instruction tuning of LLMs, promising both efficiency and resource-conscious advancements. Codes, data, and models are available: https://github.com/tianyi-lab/Cherry_LLM
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
