arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.02438cs.LGcs.AIcs.CLcs.DC
收藏

大语言模型在资源受限设备上的高效部署

Efficient Deployment of Large Language Models on Resource-constrained Devices

School of Computer Science and Technology, University of Science and Technology of China(中国科学技术大学计算机科学与技术学院);Suzhou Institute for Advanced Research, University of Science and Technology of China(中国科学技术大学苏州高等研究院)

Zhiwei Yao, Yang Xu, Hongli Xu, Yunming Liao, Zuan Xie

AI总结:

本文提出 FedSpine 联邦学习框架,融合 PEFT、结构化剪枝与在线 MAB,以自适应剪枝率和 LoRA 秩实现资源受限设备上的 LLM 高效微调部署。

摘要:

由于资源有限且数据分布异构,在资源受限(或弱性能)设备上部署大语言模型(LLM)面临重大挑战。为解决数据方面的问题,有必要利用设备端私有数据针对各类下游任务微调 LLM。尽管联邦学习(FL)提供了一种很有前景的隐私保护方案,但现有微调方法仍保留 LLM 的原始规模,导致高推理延迟和过高内存需求问题未得到解决。因此,我们设计了 FedSpine,这是一个将参数高效微调(PEFT)与结构化剪枝相结合的 FL 框架,用于在资源受限设备上高效部署 LLM。具体而言,FedSpine 引入了一个迭代过程来剪枝并微调 LLM 的参数。为缓解设备异构性的影响,该框架采用在线多臂老虎机(MAB)算法,在无需预先了解设备计算和通信能力的情况下,自适应地为异构设备确定不同的剪枝率和 LoRA 秩。因此,FedSpine 在提高微调效率的同时保持了更高的推理准确率。在包含 80 台设备的物理平台上开展的实验结果表明,与其他基线相比,在相同稀疏度水平下,FedSpine 可将微调速度提升 1.4×–6.9×,并将最终准确率提高 0.4%–4.5%。

英文摘要:

Deploying Large Language Models (LLMs) on resource-constrained (or weak) devices presents significant challenges due to limited resources and heterogeneous data distribution. To address the data concern, it is necessary to fine-tune LLMs using on-device private data for various downstream tasks. While Federated Learning (FL) offers a promising privacy-preserving solution, existing fine-tuning methods retain the original LLM size, leaving issues of high inference latency and excessive memory demands unresolved. Hence, we design FedSpine, an FL framework that combines Parameter- Efficient Fine-Tuning (PEFT) with structured pruning for efficient deployment of LLMs on resource-constrained devices. Specifically, FedSpine introduces an iterative process to prune and tune the parameters of LLMs. To mitigate the impact of device heterogeneity, an online Multi-Armed Bandit (MAB) algorithm is employed to adaptively determine different pruning ratios and LoRA ranks for heterogeneous devices without any prior knowledge of their computing and communication capabilities. As a result, FedSpine maintains higher inference accuracy while improving fine-tuning efficiency. Experimental results conducted on a physical platform with 80 devices demonstrate that FedSpine can speed up fine-tuning by 1.4$\times$-6.9$\times$ and improve final accuracy by 0.4%-4.5% under the same sparsity level compared to other baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑