arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.09739cs.CLcs.LG
收藏

QuRating:为训练语言模型选择高质量数据

QuRating: Selecting High-Quality Data for Training Language Models

Princeton University(普林斯顿大学)

Alexander Wettig, Aatmik Gupta, Saumya Malik, Danqi Chen

AI总结:

本文提出QuRating方法,利用大语言模型的成对判断训练QuRater模型,为预训练数据标注四种质量评分,据此选择数据并训练语言模型,在平衡质量与多样性后显著提升性能,并构建训练课程进一步优化。

摘要:

选择高质量的预训练数据对于创建能力强大的语言模型至关重要,但现有方法依赖于简单的启发式规则。我们引入了QuRating,一种能够捕捉人类对数据质量直觉的预训练数据选择方法。在本文中,我们研究了四种质量——写作风格、所需专业知识、事实与琐事、教育价值——并发现大语言模型能够辨别这些质量,尤其是在对文本进行成对判断时。我们训练了一个QuRater模型,从成对判断中学习标量评分,并使用它对一个260B的训练语料库按四个标准分别标注质量评分。在我们的实验中,我们根据不同的质量评分选择30B的token,并在所选数据上训练1.3B参数的语言模型。我们发现平衡质量和多样性很重要。当我们使用质量评分作为文档上的logits进行采样时,我们的模型获得了比基线更低的困惑度和更强的上下文学习性能。我们最好的模型基于教育价值,其性能与使用均匀采样训练50%更多步骤的模型相似。除了数据选择,我们还利用质量评分构建了一个训练课程,在不改变训练数据集的情况下提高了性能。我们广泛分析了质量评分,并讨论了它们的特征、偏见和更广泛的影响。

英文摘要:

Selecting high-quality pre-training data is important for creating capable language models, but existing methods rely on simple heuristics. We introduce QuRating, a method for selecting pre-training data that can capture human intuitions about data quality. In this paper, we investigate four qualities - writing style, required expertise, facts & trivia, and educational value - and find that LLMs are able to discern these qualities, especially when making pairwise judgments of texts. We train a QuRater model to learn scalar ratings from pairwise judgments, and use it to annotate a 260B training corpus with quality ratings for each of the four criteria. In our experiments, we select 30B tokens according to the different quality ratings and train 1.3B-parameter language models on the selected data. We find that it is important to balance quality and diversity. When we sample using quality ratings as logits over documents, our models obtain lower perplexity and stronger in-context learning performance than baselines. Our best model is based on educational value and performs similarly to a model trained with uniform sampling for 50% more steps. Beyond data selection, we use the quality ratings to construct a training curriculum which improves performance without changing the training dataset. We extensively analyze the quality ratings and discuss their characteristics, biases, and wider implications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑