arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.09335cs.CLcs.AI
收藏

重新思考大规模数据选择:随机选择几乎是你所需的全部

Rethinking Data Selection at Scale: Random Selection is Almost All You Need

Alibaba Group(阿里巴巴集团);School of Artificial Intelligence, Jilin University(吉林大学人工智能学院);International Center of Future Science, Jilin University(吉林大学未来科学国际中心)

Tingyu Xia, Bowen Yu, Kai Dang, An Yang, Yuan Wu, Yuan Tian, Yi Chang, Junyang Lin

AI总结:

该研究针对大规模SFT数据选择难题,复现验证多种自评分方法难超随机选择,发现数据多样性比质量更重要,提出按token长度过滤的稳定高效方法,对弱基础模型长文本训练效果显著。

摘要:

监督微调(SFT)对于使大语言模型(LLM)与人类指令对齐至关重要。SFT 期间的主要目标是从更大的数据池中选择一个小而具代表性的训练数据子集,使得用该子集进行微调能取得与使用整个数据集相当甚至更优的结果。然而,现有大多数数据选择技术都是为小规模数据池设计的,无法满足现实世界 SFT 场景的需求。本文在两百万规模的数据集上复现了几种无需外部模型辅助的自评分方法,发现处理此类大规模数据池时,几乎所有方法都难以显著优于随机选择。此外,我们的对比表明,在 SFT 过程中,数据选择的多样性比单纯关注高质量数据更为关键。我们还分析了当前几种方法的局限性,解释了它们在大规模数据集上表现不佳以及不适合此类场景的原因。最后,我们发现按 token 长度过滤数据是一种稳定且高效的效果提升方法。这种方法尤其在长文本数据上训练时,对相对较弱的基础模型(如 Llama3)极为有益。

英文摘要:

Supervised fine-tuning (SFT) is crucial for aligning Large Language Models (LLMs) with human instructions. The primary goal during SFT is to select a small yet representative subset of training data from the larger pool, such that fine-tuning with this subset achieves results comparable to or even exceeding those obtained using the entire dataset. However, most existing data selection techniques are designed for small-scale data pools, which fail to meet the demands of real-world SFT scenarios. In this paper, we replicated several self-scoring methods those that do not rely on external model assistance on two million scale datasets, and found that nearly all methods struggled to significantly outperform random selection when dealing with such large-scale data pools. Moreover, our comparisons suggest that, during SFT, diversity in data selection is more critical than simply focusing on high quality data. We also analyzed the limitations of several current approaches, explaining why they perform poorly on large-scale datasets and why they are unsuitable for such contexts. Finally, we found that filtering data by token length offers a stable and efficient method for improving results. This approach, particularly when training on long text data, proves highly beneficial for relatively weaker base models, such as Llama3.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑