arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2308.04014cs.CLcs.LG
收藏

大语言模型的持续预训练:如何重新预热你的模型?

Continual Pre-Training of Large Language Models: How to (re)warm your model?

Université de Montréal(蒙特利尔大学);Mila(米拉人工智能研究所);Eleuther AI;Concordia University(康考迪亚大学)

Kshitij Gupta, Benjamin Thérien, Adam Ibrahim, Mats L. Richter, Quentin Anthony, Eugene Belilovsky, Irina Rish, Timothée Lesort

AI总结:

本文研究大语言模型的持续预训练,通过在Pythia 410M模型上实验不同的预热策略、学习率和预热长度,发现重新预热模型虽短期增加损失,但长期显著提升下游性能,甚至优于从头训练的模型。

摘要:

大语言模型(LLMs)通常在数十亿个 token 上进行预训练,一旦有新数据可用,就只能重新开始整个过程。一种更便宜、更高效的解决方案是启用这些模型的持续预训练,即用新数据更新预训练模型,而不是从头开始重新训练。然而,新数据引起的分布偏移通常会导致在过往数据上的性能下降。为了迈向高效的持续预训练,在本研究中,我们考察了不同预热策略的效果。我们的假设是,在新数据集上训练时,必须重新提高学习率以提升计算效率。我们研究了在 Pile(上游数据,300B tokens)上预训练的模型的预热阶段,同时我们在 SlimPajama(下游数据,297B tokens)上继续进行预训练,遵循线性预热和余弦衰减计划。我们在 Pythia 410M 语言模型架构上进行了所有实验,并通过验证困惑度来评估性能。我们实验了不同的预训练检查点、各种最大学习率以及不同的预热长度。结果表明,虽然重新预热模型首先会增加上游和下游数据上的损失,但从长远来看,它能改善下游性能,甚至优于从头开始训练的模型——即使对于大型下游数据集也是如此。

英文摘要:

Large language models (LLMs) are routinely pre-trained on billions of tokens, only to restart the process over again once new data becomes available. A much cheaper and more efficient solution would be to enable the continual pre-training of these models, i.e. updating pre-trained models with new data instead of re-training them from scratch. However, the distribution shift induced by novel data typically results in degraded performance on past data. Taking a step towards efficient continual pre-training, in this work, we examine the effect of different warm-up strategies. Our hypothesis is that the learning rate must be re-increased to improve compute efficiency when training on a new dataset. We study the warmup phase of models pre-trained on the Pile (upstream data, 300B tokens) as we continue to pre-train on SlimPajama (downstream data, 297B tokens), following a linear warmup and cosine decay schedule. We conduct all experiments on the Pythia 410M language model architecture and evaluate performance through validation perplexity. We experiment with different pre-training checkpoints, various maximum learning rates, and various warmup lengths. Our results show that while rewarming models first increases the loss on upstream and downstream data, in the longer run it improves the downstream performance, outperforming models trained from scratch$\unicode{x2013}$even for a large downstream dataset.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑