arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2208.07893cs.DC
收藏

区域重叠的多服务器联邦学习收敛性研究

On the Convergence of Multi-Server Federated Learning with Overlapping Area

Zhe Qu, Xingyu Li, Jie Xu, Bo Tang, Zhuo Lu, Yao Liu

摘要:

多服务器联邦学习(FL)被认为是解决单服务器FL通信资源有限问题的一种有前途的解决方案。该架构的关键点是位于重叠区域的客户端根据所有可访问的区域模型的平均模型更新其本地模型,从而实现不同区域服务器之间的间接模型共享。由于复杂的网络拓扑结构,收敛性分析比单服务器FL更具挑战性。在本文中,我们首先提出了一种新的MS-FedAvg算法,这种多服务器FL架构,并分析其收敛性的非iid数据集一般非凸设置。由于位于每个区域服务器中的客户端数量比单服务器FL中的客户端数量少得多,因此每个客户端的带宽应该足够大以成功地与服务器通信训练模型,这表明完全客户端参与可以在多服务器FL中工作。我们提供了部分客户参与计划的收敛性分析,并开发了一种新的有偏部分参与策略,以进一步加速收敛我们的研究结果表明,收敛的结果高度依赖于在所有三种策略中的客户端的总数在每个区域类型的客户端的数量的比率。大量的实验表明,显着的性能和支持我们的理论结果。

英文摘要:

Multi-server Federated learning (FL) has been considered as a promising solution to address the limited communication resource problem of single-server FL. We consider a typical multi-server FL architecture, where the coverage areas of regional servers may overlap. The key point of this architecture is that the clients located in the overlapping areas update their local models based on the average model of all accessible regional models, which enables indirect model sharing among different regional servers. Due to the complicated network topology, the convergence analysis is much more challenging than single-server FL. In this paper, we firstly propose a novel MS-FedAvg algorithm for this multi-server FL architecture and analyze its convergence on non-iid datasets for general non-convex settings. Since the number of clients located in each regional server is much less than in single-server FL, the bandwidth of each client should be large enough to successfully communicate training models with the server, which indicates that full client participation can work in multi-server FL. Also, we provide the convergence analysis of the partial client participation scheme and develop a new biased partial participation strategy to further accelerate convergence. Our results indicate that the convergence results highly depend on the ratio of the number of clients in each area type to the total number of clients in all three strategies. The extensive experiments show remarkable performance and support our theoretical results.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑