arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2108.06453cs.LG
收藏

基于多址无线网络的高效联合元学习

Efficient Federated Meta-Learning over Multi-Access Wireless Networks

Central South University(中南大学);Tsinghua University(清华大学);University of Waterloo(滑铁卢大学)

Sheng Yue, Ju Ren, Jiang Xin, Deyu Zhang, Yaoxue Zhang, Weihua Zhuang

摘要:

联邦元学习(FML)已经成为一种很有前途的范式,以应对当今边缘学习领域的数据限制和异构性挑战。然而,它的性能往往受到收敛速度慢和相应的通信效率低的限制。此外,由于可用的无线电频谱和物联网设备的能量容量通常不足,因此在实际无线网络中部署FML时,控制资源分配和能量消耗至关重要。为了克服这些挑战,在本文中,我们严格分析了每个设备在每一轮中对全局损耗降低的贡献,并开发了一种具有非均匀设备选择方案的FML算法(称为NUFM),以加速收敛。在此之后,我们制定了一个资源分配问题,结合NUFM在多址无线系统,以共同提高收敛速度,并尽量减少挂钟时间以及能源成本。通过逐步解构原问题,设计了一种联合设备选择和资源分配策略,为解决该问题提供了理论保证。此外,我们表明,NUFM的计算复杂性可以减少从$O(d^2)$到$O(d)$(与模型维数$d$),通过结合两个一阶近似技术。大量的仿真结果表明,与现有的基线相比,所提出的方法的有效性和优越性。

英文摘要:

Federated meta-learning (FML) has emerged as a promising paradigm to cope with the data limitation and heterogeneity challenges in today's edge learning arena. However, its performance is often limited by slow convergence and corresponding low communication efficiency. In addition, since the available radio spectrum and IoT devices' energy capacity are usually insufficient, it is crucial to control the resource allocation and energy consumption when deploying FML in practical wireless networks. To overcome the challenges, in this paper, we rigorously analyze each device's contribution to the global loss reduction in each round and develop an FML algorithm (called NUFM) with a non-uniform device selection scheme to accelerate the convergence. After that, we formulate a resource allocation problem integrating NUFM in multi-access wireless systems to jointly improve the convergence rate and minimize the wall-clock time along with energy cost. By deconstructing the original problem step by step, we devise a joint device selection and resource allocation strategy to solve the problem with theoretical guarantees. Further, we show that the computational complexity of NUFM can be reduced from $O(d^2)$ to $O(d)$ (with the model dimension $d$) via combining two first-order approximation techniques. Extensive simulation results demonstrate the effectiveness and superiority of the proposed methods in comparison with existing baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑