arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2312.06353cs.LGcs.DC
收藏

联邦全参数微调十亿级语言模型,通信成本低于18千字节

Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes

College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);Alibaba Group(阿里巴巴集团)

Zhen Qin, Daoyuan Chen, Bingchen Qian, Bolin Ding, Yaliang Li, Shuiguang Deng

AI总结:

提出FedKSeed,利用零阶优化和随机种子将联邦全参数微调通信成本降至千字节级,实现十亿级LLM在设备上的高效微调,并在多个场景中优于现有方法。

摘要:

预训练大语言模型(LLMs)需要微调以提高其对自然语言指令的响应能力。联邦学习提供了一种利用终端设备上的丰富数据微调LLMs且不损害数据隐私的方法。现有的大多数联邦LLM微调方法依赖于参数高效微调技术,这些技术可能无法达到全参数微调所能实现的性能高度。然而,由于巨大的通信成本,联邦全参数微调LLMs是一个非平凡的问题。这项工作引入了FedKSeed,它采用具有有限随机种子集的零阶优化。它将服务器与客户端之间的传输需求显著减少到仅需几个随机种子和标量梯度,总计仅几千字节,使得在设备上对十亿级LLMs进行联邦全参数微调成为可能。在此基础上,我们开发了一种实现概率差异化种子采样的策略,优先考虑对模型精度影响更大的扰动。跨六个场景(涉及各种LLMs、数据集和数据划分)的实验表明,我们的方法在通信效率和新任务泛化方面均优于现有的联邦LLM微调方法。

英文摘要:

Pre-trained large language models (LLMs) need fine-tuning to improve their responsiveness to natural language instructions. Federated learning offers a way to fine-tune LLMs using the abundant data on end devices without compromising data privacy. Most existing federated fine-tuning methods for LLMs rely on parameter-efficient fine-tuning techniques, which may not reach the performance height possible with full-parameter tuning. However, federated full-parameter tuning of LLMs is a non-trivial problem due to the immense communication cost. This work introduces FedKSeed that employs zeroth-order optimization with a finite set of random seeds. It significantly reduces transmission requirements between the server and clients to just a few random seeds and scalar gradients, amounting to only a few thousand bytes, making federated full-parameter tuning of billion-sized LLMs possible on devices. Building on it, we develop a strategy enabling probability-differentiated seed sampling, prioritizing perturbations with greater impact on model accuracy. Experiments across six scenarios with various LLMs, datasets and data partitions demonstrate that our approach outperforms existing federated LLM fine-tuning methods in both communication efficiency and new task generalization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑