FedSpeed：更大的本地间隔、更少的通信轮次与更高的泛化精度
FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracy
AI总结：
本文提出 FedSpeed 方法，通过 prox 校正项和邻域梯度上升扰动缓解联邦学习中的本地偏差与过拟合问题，在减少通信轮次的同时提升泛化精度并取得 SOTA 性能。
摘要：
联邦学习是一种新兴的分布式机器学习框架，它通过大量本地设备在保护数据隐私的前提下联合训练全局模型。其性能受到本地不一致最优解所引入的不可消除偏差以及本地过拟合所导致的剧烈客户端漂移的影响。在本文中，我们提出了一种新颖且实用的方法 FedSpeed，以缓解这些问题带来的负面影响。具体而言，FedSpeed 在当前本地更新上施加 prox 校正项，以有效减少由 prox 项引入的偏差，而 prox 项是维持强本地一致性所必需的正则化器。此外，FedSpeed 将普通随机梯度与通过在邻域内额外执行一步梯度上升所计算得到的扰动进行合并，从而缓解本地过拟合问题。我们的理论分析表明，收敛速率同时与通信轮次 $T$ 和本地间隔 $K$ 相关，并且在设置合适的本地间隔时，其收敛上界为 $\small \mathcal{O}(1/T)$。此外，我们在真实世界数据集上进行了大量实验，以验证所提出的 FedSpeed 的有效性。实验结果表明，在通用联邦学习实验设置下，FedSpeed 的收敛速度显著更快，并且相较于多个基线方法取得了最先进的（SOTA）性能。我们的代码已开源，地址为 \url{https://github.com/woodenchild95/FL-Simulator.git}。
英文摘要：
Federated learning is an emerging distributed machine learning framework which jointly trains a global model via a large number of local devices with data privacy protections. Its performance suffers from the non-vanishing biases introduced by the local inconsistent optimal and the rugged client-drifts by the local over-fitting. In this paper, we propose a novel and practical method, FedSpeed, to alleviate the negative impacts posed by these problems. Concretely, FedSpeed applies the prox-correction term on the current local updates to efficiently reduce the biases introduced by the prox-term, a necessary regularizer to maintain the strong local consistency. Furthermore, FedSpeed merges the vanilla stochastic gradient with a perturbation computed from an extra gradient ascent step in the neighborhood, thereby alleviating the issue of local over-fitting. Our theoretical analysis indicates that the convergence rate is related to both the communication rounds $T$ and local intervals $K$ with a upper bound $\small \mathcal{O}(1/T)$ if setting a proper local interval. Moreover, we conduct extensive experiments on the real-world dataset to demonstrate the efficiency of our proposed FedSpeed, which performs significantly faster and achieves the state-of-the-art (SOTA) performance on the general FL experimental settings than several baselines. Our code is available at \url{https://github.com/woodenchild95/FL-Simulator.git}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
