arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.01353cs.LGcs.AIcs.CL
收藏

高效蒸馏大语言模型以用于边缘应用

Efficiently Distilling LLMs for Edge Applications

IBM Research(IBM研究院)

Achintya Kundu, Fabian Lim, Aaron Chew, Laura Wynter, Penny Chong, Rhui Dih Lee

AI总结:

本文提出MLFS方法,通过多阶段低秩微调实现参数高效的大语言模型超网训练,在恒定成本下生成适配边缘应用的多规格模型,并验证了编码器模型的高质量及解码器切片对训练时间的显著缩减。

摘要:

大语言模型的超网训练在工业应用中备受关注,因为它能够以恒定成本生成一系列较小的模型,无论生成多少个(不同规模/延迟的)模型。我们提出了一种名为超Transformer多阶段低秩微调(MLFS)的新方法,用于参数高效的超网训练。我们证明,可以获得适合商业边缘应用的高质量编码器模型,并且尽管仅解码器模型对同等程度的压缩具有抵抗性,但解码器可以被有效切片以显著减少训练时间。

英文摘要:

Supernet training of LLMs is of great interest in industrial applications as it confers the ability to produce a palette of smaller models at constant cost, regardless of the number of models (of different size / latency) produced. We propose a new method called Multistage Low-rank Fine-tuning of Super-transformers (MLFS) for parameter-efficient supernet training. We show that it is possible to obtain high-quality encoder models that are suitable for commercial edge applications, and that while decoder-only models are resistant to a comparable degree of compression, decoders can be effectively sliced for a significant reduction in training time.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑