arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2108.05765cs.LGcs.DC
收藏

基于动态注意力的通信高效联邦学习

Dynamic Attention-based Communication-Efficient Federated Learning

Singapore University of Technology and Design(新加坡科技设计大学);National University of Singapore(新加坡国立大学)

Zihan Chen, Kai Fong Ernest Chong, Tony Q. S. Quek

摘要:

联邦学习(FL)提供了一种解决方案,用于训练全局机器学习模型,同时仍然保持数据隐私,而无需访问本地存储在客户端的数据。然而,当客户端数据分发是非IID时,FL遭受性能降级,并且由于通信限制,较长的训练持续时间来对抗这种降级可能不一定可行。为了解决这一挑战,我们提出了一种新的自适应训练算法$\texttt{AdaFL}$,它包括两个组成部分:(i)一个基于注意力的客户端选择机制的客户端之间的一个更公平的训练方案;和(ii)一个动态分数的方法来平衡性能稳定性和通信效率之间的权衡。实验结果表明,$\texttt{AdaFL}$算法在模型精度、性能稳定性和通信效率三个方面均优于常用的$\texttt{FedAvg}$算法,并可用于进一步改进现有的FL算法。

英文摘要:

Federated learning (FL) offers a solution to train a global machine learning model while still maintaining data privacy, without needing access to data stored locally at the clients. However, FL suffers performance degradation when client data distribution is non-IID, and a longer training duration to combat this degradation may not necessarily be feasible due to communication limitations. To address this challenge, we propose a new adaptive training algorithm $\texttt{AdaFL}$, which comprises two components: (i) an attention-based client selection mechanism for a fairer training scheme among the clients; and (ii) a dynamic fraction method to balance the trade-off between performance stability and communication efficiency. Experimental results show that our $\texttt{AdaFL}$ algorithm outperforms the usual $\texttt{FedAvg}$ algorithm, and can be incorporated to further improve various state-of-the-art FL algorithms, with respect to three aspects: model accuracy, performance stability, and communication efficiency.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑