面向语言模型的最优控制数据选择方法
Data Selection via Optimal Control for Language Models
AI总结:
该研究将语言模型预训练数据选择建模为最优控制问题,基于庞特里亚金极大值原理提出PDS框架,可高效筛选高质量数据,提升多规模模型下游性能并降低数据需求。
摘要:
本研究旨在从海量语料中筛选高质量预训练数据,以提升语言模型(LM)的下游任务能力。我们将数据选择问题建模为广义最优控制问题,可通过庞特里亚金极大值原理(Pontryagin's Maximum Principle, PMP)进行理论求解,推导得到一组刻画最优数据选择与LM训练动态之间关系的必要条件。基于这些理论结果,我们提出了基于PMP的数据选择框架(PMP-based Data Selection, PDS),通过求解PMP条件来近似实现最优数据选择。实验中,我们采用PDS从CommonCrawl数据集中筛选数据,结果显示PDS筛选出的语料能加快LM的学习速度,且在多种模型规模下,持续提升模型在广泛下游任务上的性能。此外,根据缩放定律(Scaling Laws)对测试损失曲线的外推结果表明,PDS的增益可拓展至在约10T tokens上训练的约400B参数规模的模型。当预训练数据有限时,PDS还能提升数据利用率,将数据需求量降低1.8倍,有助于缓解网页爬取语料快速耗尽的问题。我们的代码、模型和数据可在https://github.com/microsoft/LMOps/tree/main/data_selection获取。
英文摘要:
This work investigates the selection of high-quality pre-training data from massive corpora to enhance LMs' capabilities for downstream usage. We formulate data selection as a generalized Optimal Control problem, which can be solved theoretically by Pontryagin's Maximum Principle (PMP), yielding a set of necessary conditions that characterize the relationship between optimal data selection and LM training dynamics. Based on these theoretical results, we introduce PMP-based Data Selection (PDS), a framework that approximates optimal data selection by solving the PMP conditions. In our experiments, we adopt PDS to select data from CommmonCrawl and show that the PDS-selected corpus accelerates the learning of LMs and constantly boosts their performance on a wide range of downstream tasks across various model sizes. Moreover, the benefits of PDS extend to ~400B models trained on ~10T tokens, as evidenced by the extrapolation of the test loss curves according to the Scaling Laws. PDS also improves data utilization when the pre-training data is limited, by reducing the data demand by 1.8 times, which helps mitigate the quick exhaustion of available web-crawled corpora. Our code, model, and data can be found at https://github.com/microsoft/LMOps/tree/main/data_selection.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。