面向高效大语言模型指令微调的重要性感知数据选择
Importance-Aware Data Selection for Efficient LLM Instruction Tuning
AI总结：
本文提出模型指令弱点值（MIWV）指标，利用上下文学习中的回答差异量化指令数据对模型能力提升的重要性，并证明仅用前1%数据微调即可超越全量数据训练效果。
摘要：
指令微调在提升大语言模型（LLMs）的性能与效率方面发挥着关键作用。其成功不仅取决于指令数据的质量，还取决于大语言模型本身的固有能力。一些研究表明，即使使用少量高质量数据，也能取得与使用全量数据集相当甚至更优的指令微调效果。然而，与其仅仅专注于计算数据质量分数来评估指令数据，当前越来越需要选择能够最大程度提升给定大语言模型指令微调性能的高质量数据。在本文中，我们提出模型指令弱点值（Model Instruction Weakness Value，MIWV）作为一种新的度量指标，用于量化指令数据在增强模型能力方面的重要性。MIWV 指标来源于模型在使用上下文学习（In-Context Learning，ICL）时回答的差异，有助于识别对提升指令微调性能最有益的数据。我们的实验结果表明，仅选择基于 MIWV 排名前 1% 的数据进行训练，即可优于在完整数据集上的训练效果。此外，该方法超越了现有专注于数据质量评分以进行数据选择的研究，为所提方法的有效性提供了有力的经验证据。
英文摘要：
Instruction tuning plays a critical role in enhancing the performance and efficiency of Large Language Models (LLMs). Its success depends not only on the quality of the instruction data but also on the inherent capabilities of the LLM itself. Some studies suggest that even a small amount of high-quality data can achieve instruction fine-tuning results that are on par with, or even exceed, those from using a full-scale dataset. However, rather than focusing solely on calculating data quality scores to evaluate instruction data, there is a growing need to select high-quality data that maximally enhances the performance of instruction tuning for a given LLM. In this paper, we propose the Model Instruction Weakness Value (MIWV) as a novel metric to quantify the importance of instruction data in enhancing model's capabilities. The MIWV metric is derived from the discrepancies in the model's responses when using In-Context Learning (ICL), helping identify the most beneficial data for enhancing instruction tuning performance. Our experimental results demonstrate that selecting only the top 1\% of data based on MIWV can outperform training on the full dataset. Furthermore, this approach extends beyond existing research that focuses on data quality scoring for data selection, offering strong empirical evidence supporting the effectiveness of our proposed method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
