arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.18779cs.LGcs.CL
收藏

一点帮助大有可为:利用小型语言模型实现高效的 LLM 训练

A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs

Google Research(谷歌研究院);Google DeepMind(谷歌DeepMind)
Ankit Singh Rawat, Veeranjaneyulu Sadhanala, Afshin Rostamizadeh, Ayan Chakrabarti, Wittawat Jitkrittum, Vladimir Feinberg, Seungyeon Kim, Hrayr Harutyunyan, Ni… 展开作者

Ankit Singh Rawat, Veeranjaneyulu Sadhanala, Afshin Rostamizadeh, Ayan Chakrabarti, Wittawat Jitkrittum, Vladimir Feinberg, Seungyeon Kim, Hrayr Harutyunyan, Nikunj Saunshi, Zachary Nado, Rakesh Shivanna, Sashank J. Reddi, Aditya Krishna Menon, Rohan Anil, Sanjiv Kumar

AI总结:

本文提出利用小型语言模型提供软标签并筛选高价值训练样本,以提升大型语言模型预训练效率与质量,并通过统计框架和 Pile 实验验证其有效性。

摘要:

大型语言模型(LLM)开发中的一个主要挑战是其繁重的预训练成本。通常,此类预训练涉及在大型语料库上优化自监督目标(例如下一 token 预测)。本文探索了一种有前景的范式,通过适当地利用小型语言模型(SLM)来提高 LLM 预训练的效率和质量。具体而言,该范式依赖 SLM 同时完成两项任务:(1)提供软标签作为额外的训练监督;(2)选择一小部分有价值的(“信息丰富”且“困难”的)训练样本。综合起来,这使得 SLM 的预测分布能够有效迁移到 LLM,同时优先关注训练数据分布中的特定区域。实验表明,与标准训练相比,这减少了 LLM 的训练时间,同时提高了整体质量。理论上,我们开发了一个统计框架,用于系统研究 SLM 在实现高质量 LLM 高效训练方面的效用。特别是,我们的框架刻画了 SLM 看似低质量的监督如何能够增强能力强得多的 LLM 的训练。此外,该框架还强调需要自适应地利用此类监督,在 SLM 提供的软标签所引入的偏差和方差之间取得平衡。我们在 Pile 数据集上,利用一个拥有 15 亿参数的较小 LM 来改进一个拥有 28 亿参数的 LLM 的预训练,从而证实了我们的理论框架。

英文摘要:

A primary challenge in large language model (LLM) development is their onerous pre-training cost. Typically, such pre-training involves optimizing a self-supervised objective (such as next-token prediction) over a large corpus. This paper explores a promising paradigm to improve LLM pre-training efficiency and quality by suitably leveraging a small language model (SLM). In particular, this paradigm relies on an SLM to both (1) provide soft labels as additional training supervision, and (2) select a small subset of valuable ("informative" and "hard") training examples. Put together, this enables an effective transfer of the SLM's predictive distribution to the LLM, while prioritizing specific regions of the training data distribution. Empirically, this leads to reduced LLM training time compared to standard training, while improving the overall quality. Theoretically, we develop a statistical framework to systematically study the utility of SLMs in enabling efficient training of high-quality LLMs. In particular, our framework characterizes how the SLM's seemingly low-quality supervision can enhance the training of a much more capable LLM. Furthermore, it also highlights the need for an adaptive utilization of such supervision, by striking a balance between the bias and variance introduced by the SLM-provided soft labels. We corroborate our theoretical framework by improving the pre-training of an LLM with 2.8B parameters by utilizing a smaller LM with 1.5B parameters on the Pile dataset.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑