后期到早期训练:LET LLMs 早期学习,从而更快且更好
Late-to-Early Training: LET LLMs Learn Earlier, So Faster and Better
AI总结:
本文提出LET范式,通过利用已有小型预训练模型加速大模型训练,实现更快的训练速度和更高的性能。
摘要:
随着大型语言模型(LLMs)通过扩展模型和数据规模实现显著的实证成功,预训练已成为越来越关键却计算上极具挑战性的过程,阻碍了快速的发展。尽管已有大量预训练的LLM在显著的计算成本下开发,一个基本的现实问题仍未得到充分探讨:我们可以利用已有的小型预训练模型来加速更大模型的训练吗?在本文中,我们提出了一种后期到早期训练(LET)范式,使LLMs能够在早期步骤和层中显式学习后期知识。核心思想是利用预训练(即后期训练阶段)模型的后期层的表示来指导LLM的早期层的早期训练。我们识别出两个驱动LET有效性的关键机制:后期到早期步骤学习和后期到早期层学习。这些机制显著加速了训练收敛,同时稳健地增强了语言建模能力和下游任务性能,使训练更快且性能更优。在14亿和70亿参数模型上的大量实验展示了LET的效率和有效性。值得注意的是,当在Pile数据集上训练14亿参数的LLM时,我们的方法在使用比目标模型少10倍参数的预训练模型的情况下,实现了1.6倍的加速,下游任务准确率提高了近5%。
英文摘要:
As Large Language Models (LLMs) achieve remarkable empirical success through scaling model and data size, pretraining has become increasingly critical yet computationally prohibitive, hindering rapid development. Despite the availability of numerous pretrained LLMs developed at significant computational expense, a fundamental real-world question remains underexplored: \textit{Can we leverage existing small pretrained models to accelerate the training of larger models?} In this paper, we propose a Late-to-Early Training (LET) paradigm that enables LLMs to explicitly learn later knowledge in earlier steps and earlier layers. The core idea is to guide the early layers of an LLM during early training using representations from the late layers of a pretrained (i.e. late training phase) model. We identify two key mechanisms that drive LET's effectiveness: late-to-early-step learning and late-to-early-layer learning. These mechanisms significantly accelerate training convergence while robustly enhancing both language modeling capabilities and downstream task performance, enabling faster training with superior performance. Extensive experiments on 1.4B and 7B parameter models demonstrate LET's efficiency and effectiveness. Notably, when training a 1.4B LLM on the Pile dataset, our method achieves up to 1.6$\times$ speedup with nearly 5\% improvement in downstream task accuracy compared to standard training, even when using a pretrained model with 10$\times$ fewer parameters than the target model.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。