arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.16682cs.CLcs.LG
收藏

提升大语言模型训练稳定性的方法

Methods of improving LLM training stability

NVIDIA(英伟达)

Oleg Rybakov, Mike Chrzanowski, Peter Dykas, Jinze Xue, Ben Lanir

AI总结:

本文针对大语言模型训练不稳定问题,通过分析Transformer各线性层输出增长,提出在Proj、FC2或QKV层后施加层归一化及结合softmax capping的方法,成功将学习率提升1.5倍并显著降低困惑度。

摘要:

大语言模型(LLMs)的训练稳定性是一个重要的研究课题。复现训练不稳定现象可能代价高昂,因此我们使用一个具有830M参数的小型语言模型,并尝试使用更高的学习率来迫使模型发散。训练不稳定的来源之一是注意力层中logits的增长。我们扩展了先前工作的关注点,不仅关注logits的幅度,还关注Transformer块中线性层的所有输出。我们观察到,在高学习率下,所有线性层输出的L2范数会随着每个训练步骤而增长,并且模型会发散。具体而言,我们观察到QKV、Proj和FC2层的输出幅度增长最大。这促使我们探索几种方案:1)不仅在QK层之后应用层归一化,还在Proj和FC2层之后也应用层归一化;2)在QKV层之后应用层归一化(并移除预归一化);3)将QK层归一化与softmax capping结合使用。我们表明,与仅基于QK层归一化的方法相比,使用后两种方法可以在不导致模型发散的情况下将学习率提高1.5倍。此外,与基线模型相比,我们观察到所有三种方法在困惑度上都有显著改善。

英文摘要:

Training stability of large language models(LLMs) is an important research topic. Reproducing training instabilities can be costly, so we use a small language model with 830M parameters and experiment with higher learning rates to force models to diverge. One of the sources of training instability is the growth of logits in attention layers. We extend the focus of the previous work and look not only at the magnitude of the logits but at all outputs of linear layers in the Transformer block. We observe that with a high learning rate the L2 norm of all linear layer outputs can grow with each training step and the model diverges. Specifically we observe that QKV, Proj and FC2 layers have the largest growth of the output magnitude. This prompts us to explore several options: 1) apply layer normalization not only after QK layers but also after Proj and FC2 layers too; 2) apply layer normalization after the QKV layer (and remove pre normalization). 3) apply QK layer normalization together with softmax capping. We show that with the last two methods we can increase learning rate by 1.5x (without model divergence) in comparison to an approach based on QK layer normalization only. Also we observe significant perplexity improvements for all three methods in comparison to the baseline model.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑