arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.16405cs.CLcs.AIcs.LG
收藏

将稀疏微调扩展到大语言模型

Scaling Sparse Fine-Tuning to Large Language Models

University of Cambridge(剑桥大学);University of Edinburgh(爱丁堡大学)

Alan Ansell, Ivan Vulić, Hannah Sterz, Anna Korhonen, Edoardo M. Ponti

AI总结:

针对大语言模型全量微调成本高、现有稀疏微调内存随模型规模增长的问题,提出新型稀疏微调方法SpIEL,在指令微调任务中性能优于LoRA且运行时间相当,还兼容量化与高效优化器以支撑更大规模模型。

摘要:

大语言模型(LLM)由于参数量极为庞大,难以进行全量微调(例如通过指令或人类反馈微调)。一类参数高效的稀疏微调方法已被证明在性能上颇具潜力,但其内存需求会随LLM的规模成比例增长。在本研究中,我们将稀疏微调扩展至LLaMA 2 7B和13B等最先进的LLM。我们提出了SpIEL这一新型稀疏微调方法,针对目标密度水平,该方法维护一个参数索引数组,以及这些参数相对于其预训练值的增量。该方法迭代执行三个步骤:(a)更新活跃增量,(b)(根据增量的幅度变化)剪枝索引,(c)索引重新生长。对于重新生长,我们探索了两种准则,分别基于少量候选参数的累积梯度,或使用高效SM3优化器估计的近似动量。我们在标准数据集混合集上开展了LLM的指令微调实验,发现SpIEL在性能上通常优于LoRA(低秩适配)等流行的参数高效微调方法,且运行时间与之相当。我们还证明SpIEL可同时兼容量化和高效优化器,便于扩展到更大规模的模型。我们在https://github.com/AlanAnsell/peft发布了SpIEL的代码,在https://github.com/ducdauge/sft-llm发布了指令微调实验的代码。

英文摘要:

Large Language Models (LLMs) are difficult to fully fine-tune (e.g., with instructions or human feedback) due to their sheer number of parameters. A family of parameter-efficient sparse fine-tuning methods have proven promising in terms of performance but their memory requirements increase proportionally to the size of the LLMs. In this work, we scale sparse fine-tuning to state-of-the-art LLMs like LLaMA 2 7B and 13B. We propose SpIEL, a novel sparse fine-tuning method which, for a desired density level, maintains an array of parameter indices and the deltas of these parameters relative to their pretrained values. It iterates over: (a) updating the active deltas, (b) pruning indices (based on the change of magnitude of their deltas) and (c) regrowth of indices. For regrowth, we explore two criteria based on either the accumulated gradients of a few candidate parameters or their approximate momenta estimated using the efficient SM3 optimizer. We experiment with instruction-tuning of LLMs on standard dataset mixtures, finding that SpIEL is often superior to popular parameter-efficient fine-tuning methods like LoRA (low-rank adaptation) in terms of performance and comparable in terms of run time. We additionally show that SpIEL is compatible with both quantization and efficient optimizers, to facilitate scaling to ever-larger model sizes. We release the code for SpIEL at https://github.com/AlanAnsell/peft and for the instruction-tuning experiments at https://github.com/ducdauge/sft-llm.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑