FedDA：基于重启动对偶平均的局部自适应梯度法快速框架
FedDA: Faster Framework of Local Adaptive Gradient Methods via Restarted Dual Averaging
摘要：
联邦学习（Federated learning，FL）是一种新兴的处理大规模分布式数据的学习范式。在联邦学习中，一组客户端在服务器的协调下联合执行机器学习任务。FedAvg算法是解决联邦学习问题最广泛使用的方法之一。在FedAvg中，学习率是一个常数，而不是自适应地变化。自适应梯度方法表现出优越的性能超过恒定的学习率调度，但是，仍然没有一个通用的框架，将自适应梯度方法纳入联邦设置。在本文中，我们提出了\textbf{FedDA}，一个新的框架，局部自适应梯度方法。该框架采用了重新启动的双平均技术，具有灵活的各种梯度估计方法和自适应学习率公式。特别是，我们分析\textbf{FedDA-MVR}，我们的框架的实例，并表明，它实现梯度复杂性$\tilde{O}(ε^{-1.5})$和通信复杂性$\tilde{O}(ε^{-1})$找到一个固定点$ε$。这与一阶FL算法的最佳已知速率相匹配，并且\textbf{FedDA-MVR}是实现该速率的第一个自适应FL算法。我们还进行了大量的数值实验，以验证我们的方法的有效性。
英文摘要：
Federated learning (FL) is an emerging learning paradigm to tackle massively distributed data. In Federated Learning, a set of clients jointly perform a machine learning task under the coordination of a server. The FedAvg algorithm is one of the most widely used methods to solve Federated Learning problems. In FedAvg, the learning rate is a constant rather than changing adaptively. The adaptive gradient methods show superior performance over the constant learning rate schedule; however, there is still no general framework to incorporate adaptive gradient methods into the federated setting. In this paper, we propose \textbf{FedDA}, a novel framework for local adaptive gradient methods. The framework adopts a restarted dual averaging technique and is flexible with various gradient estimation methods and adaptive learning rate formulations. In particular, we analyze \textbf{FedDA-MVR}, an instantiation of our framework, and show that it achieves gradient complexity $\tilde{O}(ε^{-1.5})$ and communication complexity $\tilde{O}(ε^{-1})$ for finding a stationary point $ε$. This matches the best known rate for first-order FL algorithms and \textbf{FedDA-MVR} is the first adaptive FL algorithm that achieves this rate. We also perform extensive numerical experiments to verify the efficacy of our method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
