arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2109.10903cs.ITcs.NImath.IT
收藏

基于无线边缘网络的大规模联邦学习网内计算

In-network Computation for Large-scale Federated Learning over Wireless Edge Networks

Thinh Quang Dinh, Diep N. Nguyen, Dinh Thai Hoang, Pham Tran Vu, Eryk Dutkiewicz

摘要:

大多数传统的联邦学习(FL)模型使用星形网络拓扑,其中所有用户在单个服务器处聚合他们的本地模型(例如,云服务器)。这在服务器的通信和计算方面都造成了显著的开销,延迟了训练过程,特别是对于具有离散节点的大规模FL系统。本文提出了一种新的边缘网络架构,可以在服务器上分散模型聚合过程,从而显着降低整个FL网络的训练延迟。具体而言,我们设计了一个高效的网络计算协议(INC)组成的用户调度机制,网络聚合过程(INA),这是专为原始和原始对偶方法在分布式机器学习问题,和一个网络路由算法。在建议的INA,我们然后制定一个联合路由和资源优化问题,旨在最大限度地减少聚合延迟。该问题是NP-难的,因此我们提出了一个多项式时间路由算法,可以达到接近最优性能的理论界。仿真结果表明,所提出的INC框架不仅可以帮助减少FL训练延迟,高达5.6倍,而且还显着减少云的流量和计算开销。这可以实现大规模FL。

英文摘要:

Most conventional Federated Learning (FL) models are using a star network topology where all users aggregate their local models at a single server (e.g., a cloud server). That causes significant overhead in terms of both communications and computing at the server, delaying the training process, especially for large scale FL systems with straggling nodes. This paper proposes a novel edge network architecture that enables decentralizing the model aggregation process at the server, thereby significantly reducing the training delay for the whole FL network. Specifically, we design a highly-effective in-network computation protocol (INC) consisting of a user scheduling mechanism, an in-network aggregation process (INA) which is designed for both primal- and primal-dual methods in distributed machine learning problems, and a network routing algorithm. Under the proposed INA, we then formulate a joint routing and resource optimization problem, aiming to minimize the aggregation latency. The problem is NP-hard, and thus we propose a polynomial time routing algorithm which can achieve near optimal performance with a theoretical bound. Simulation results showed that the proposed INC framework can not only help reduce the FL training latency, up to 5.6 times, but also significantly decrease cloud's traffic and computing overhead. This can enable large-scale FL.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑