通过策划LLM驱动的评分系统提升数据效率
Improving Data Efficiency via Curating LLM-Driven Rating Systems
AI总结：
提出DS2方法，通过分数转移矩阵纠正LLM评分偏差并促进数据多样性，仅用3.3%的数据即超越全规模数据集，挑战了传统数据缩放定律。
摘要：
指令微调对于将大型语言模型（LLMs）适配到下游任务至关重要，最近的研究表明，少量人工策划的数据可以超越更大的数据集，这挑战了传统的数据缩放定律。虽然基于LLM的数据质量评分系统提供了一种成本效益高的人类标注替代方案，但它们通常存在不准确和偏差，即使在像GPT-4这样的强大模型中也是如此。在这项工作中，我们引入了DS2，一种用于数据选择的多样性感知分数策划方法。通过分数转移矩阵系统地建模误差模式，DS2纠正了基于LLM的分数，并促进了所选数据样本的多样性。我们的方法表明，一个策划的子集（仅占原始数据集的3.3%）在各种机器对齐基准上超越了全规模数据集（300k样本），并且在相同样本量（1k样本）下匹配或超越了像LIMA这样的人类对齐数据集。这些发现挑战了传统的数据缩放假设，强调了冗余、低质量的样本会降低性能，并重申了“多即是少”。
英文摘要：
Instruction tuning is critical for adapting large language models (LLMs) to downstream tasks, and recent studies have demonstrated that small amounts of human-curated data can outperform larger datasets, challenging traditional data scaling laws. While LLM-based data quality rating systems offer a cost-effective alternative to human annotation, they often suffer from inaccuracies and biases, even in powerful models like GPT-4. In this work, we introduce DS2, a Diversity-aware Score curation method for Data Selection. By systematically modeling error patterns through a score transition matrix, DS2 corrects LLM-based scores and promotes diversity in the selected data samples. Our approach shows that a curated subset (just 3.3% of the original dataset) outperforms full-scale datasets (300k samples) across various machine-alignment benchmarks, and matches or surpasses human-aligned datasets such as LIMA with the same sample size (1k samples). These findings challenge conventional data scaling assumptions, highlighting that redundant, low-quality samples can degrade performance and reaffirming that "more can be less."
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
