arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2204.11536cs.DCcs.AI
收藏

FedDUAP:服务器上使用共享数据的动态更新和自适应剪枝的联邦学习

FedDUAP: Federated Learning with Dynamic Update and Adaptive Pruning Using Shared Data on the Server

Soochow University(苏州大学);Baidu Inc.(百度公司);Auburn University(奥本大学);North Carolina State University(北卡罗来纳州立大学)

Hong Zhang, Ji Liu, Juncheng Jia, Yang Zhou, Huaiyu Dai, Dejing Dou

摘要:

尽管联邦学习(FL)取得了卓越的性能,但它面临着两个关键挑战,即,计算资源有限,训练效率低。在本文中,我们提出了一种新的FL框架,即,FedDUAP,有两个原创贡献,利用服务器上的不敏感数据和边缘设备上的分散数据,进一步提高训练效率。首先,设计了一种动态服务器更新算法,利用服务器上的不敏感数据,动态确定服务器更新的最优步长,提高全局模型的收敛性和精度。其次,提出了一种分层自适应的模型剪枝方法,对不同维度和重要性的多层模型进行独特的剪枝操作,在效率和效果之间取得了较好的平衡。通过将两种原始技术整合在一起,我们提出的FL模型FedDUAP在准确性(高出4.8%),效率(快2.8倍)和计算成本(小61.9%)方面明显优于基线方法。

英文摘要:

Despite achieving remarkable performance, Federated Learning (FL) suffers from two critical challenges, i.e., limited computational resources and low training efficiency. In this paper, we propose a novel FL framework, i.e., FedDUAP, with two original contributions, to exploit the insensitive data on the server and the decentralized data in edge devices to further improve the training efficiency. First, a dynamic server update algorithm is designed to exploit the insensitive data on the server, in order to dynamically determine the optimal steps of the server update for improving the convergence and accuracy of the global model. Second, a layer-adaptive model pruning method is developed to perform unique pruning operations adapted to the different dimensions and importance of multiple layers, to achieve a good balance between efficiency and effectiveness. By integrating the two original techniques together, our proposed FL model, FedDUAP, significantly outperforms baseline approaches in terms of accuracy (up to 4.8% higher), efficiency (up to 2.8 times faster), and computational cost (up to 61.9% smaller).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑