arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2211.09320cs.DC
收藏

基于全局动量融合的梯度压缩联邦学习通信效率研究

Improving Federated Learning Communication Efficiency with Global Momentum Fusion for Gradient Compression Schemes

Chun-Chih Kuo, Ted Tsei Kuo, Chia-Yu Lin

摘要:

联邦学习中的通信成本阻碍了系统从更多客户端获取更多数据的可扩展性。拟议的FL采用轴辐式网络拓扑结构。所有客户端都通过中央服务器进行通信。因此,已经提出了通过诸如数据压缩的技术来减少通信开销以缓解该问题。联邦学习的另一个挑战是不平衡的数据分布,在典型的联邦学习设置中,每个客户端上的数据不是独立和相同分布的(非IID)。在本文中,我们提出了一种新的压缩补偿方案,称为全球动量融合(GMF),减少FL客户端和服务器之间的通信开销,并保持可比的模型精度在非IID数据的存在。GitHub资源库:https://github.com/tony92151/global-momentum-fusion-fl

英文摘要:

Communication costs within Federated learning hinder the system scalability for reaching more data from more clients. The proposed FL adopts a hub-and-spoke network topology. All clients communicate through the central server. Hence, reducing communication overheads via techniques such as data compression has been proposed to mitigate this issue. Another challenge of federated learning is unbalanced data distribution, data on each client are not independent and identically distributed (non-IID) in a typical federated learning setting. In this paper, we proposed a new compression compensation scheme called Global Momentum Fusion (GMF) which reduces communication overheads between FL clients and the server and maintains comparable model accuracy in the presence of non-IID data. GitHub repository: https://github.com/tony92151/global-momentum-fusion-fl

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑