arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.05331cs.DCcs.NI
收藏

面向去中心化联邦学习的实用覆盖网络

Towards Practical Overlay Networks for Decentralized Federated Learning

Yifan Hua, Jinlong Pang, Xiaoxue Zhang, Yi Liu, Xiaofeng Shi, Bao Wang, Yang Liu, Chen Qian

AI总结:

针对去中心化联邦学习缺乏可实际部署的覆盖网络构建与维护协议的问题,本文提出 FedLay,以去中心化方式构建并维护近随机正则拓扑,在真实数据集上实现更快收敛、更高准确率、低通信开销及节点动态容错。

摘要:

去中心化联邦学习(DFL)利用点对点通信来避免联邦学习中的单点故障问题,并被认为是在分布式设备上执行机器学习任务的一种有吸引力的解决方案。我们针对 DFL 的一个基础性网络问题给出了首个解决方案:DFL 应当使用何种覆盖网络,才能实现高准确率模型的快速训练、低通信量,以及去中心化的构建与维护?已有研究考察了 DFL 的覆盖拓扑,但现有 DFL 拓扑均不包含用于网络构建和拓扑维护的去中心化协议。缺少这些协议,DFL 就无法在实践中运行。本工作提出了一个名为 FedLay 的覆盖网络,为实用 DFL 提供快速训练和较低的通信成本。FedLay 是首个以去中心化方式构建近随机正则拓扑,并在节点加入和故障情况下维护这些拓扑的解决方案。基于原型实现和仿真的实验表明,与现有 DFL 解决方案相比,FedLay 在真实数据集上实现了最快的模型收敛和最高的准确率,同时只产生较小的通信成本,并且能够承受节点加入和故障。

英文摘要:

Decentralized federated learning (DFL) uses peer-to-peer communication to avoid the single point of failure problem in federated learning and has been considered an attractive solution for machine learning tasks on distributed devices. We provide the first solution to a fundamental network problem of DFL: what overlay network should DFL use to achieve fast training of highly accurate models, low communication, and decentralized construction and maintenance? Overlay topologies of DFL have been investigated, but no existing DFL topology includes decentralized protocols for network construction and topology maintenance. Without these protocols, DFL cannot run in practice. This work presents an overlay network, called FedLay, which provides fast training and low communication cost for practical DFL. FedLay is the first solution for constructing near-random regular topologies in a decentralized manner and maintaining the topologies under node joins and failures. Experiments based on prototype implementation and simulations show that FedLay achieves the fastest model convergence and highest accuracy on real datasets compared to existing DFL solutions while incurring small communication costs and being resilient to node joins and failures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑