分布式非凸优化在间歇性客户端可用性下的次线性加速
Distributed Non-Convex Optimization with Sublinear Speedup under Intermittent Client Availability
AI总结：
针对移动环境中客户端间歇性可用导致联邦学习性能下降的问题，提出FedLaAvg算法，利用所有客户端最新梯度更新全局模型，实现次线性加速，并在MNIST和Sentiment140上验证了稳定性。
摘要：
联邦学习是一种新的分布式机器学习框架，其中一组异构客户端在不共享训练数据的情况下协作训练模型。在这项工作中，我们考虑了在移动环境中部署联邦学习时一个实际且普遍存在的问题：间歇性客户端可用性，即符合条件的客户端集合可能在训练过程中发生变化。这种间歇性客户端可用性会严重恶化经典联邦平均算法（简称FedAvg）的性能。因此，我们提出了一种简单的分布式非凸优化算法，称为联邦最新平均（简称FedLaAvg），该算法利用所有客户端的最新梯度，即使客户端不可用，也用于每次迭代中联合更新全局模型。我们的理论分析表明，FedLaAvg达到了$O(E^{1/2}/(N^{1/4} T^{1/2}))$的收敛速率，相对于总客户端数量实现了次线性加速。我们实现了FedLaAvg及若干基线，并在基准MNIST和Sentiment140数据集上进行了评估。评估结果表明，在凸和非凸设置下，FedLaAvg比FedAvg实现了更稳定的训练，并且确实达到了次线性加速。
英文摘要：
Federated learning is a new distributed machine learning framework, where a bunch of heterogeneous clients collaboratively train a model without sharing training data. In this work, we consider a practical and ubiquitous issue when deploying federated learning in mobile environments: intermittent client availability, where the set of eligible clients may change during the training process. Such intermittent client availability would seriously deteriorate the performance of the classical Federated Averaging algorithm (FedAvg for short). Thus, we propose a simple distributed non-convex optimization algorithm, called Federated Latest Averaging (FedLaAvg for short), which leverages the latest gradients of all clients, even when the clients are not available, to jointly update the global model in each iteration. Our theoretical analysis shows that FedLaAvg attains the convergence rate of $O(E^{1/2}/(N^{1/4} T^{1/2}))$, achieving a sublinear speedup with respect to the total number of clients. We implement FedLaAvg along with several baselines and evaluate them over the benchmarking MNIST and Sentiment140 datasets. The evaluation results demonstrate that FedLaAvg achieves more stable training than FedAvg in both convex and non-convex settings and indeed reaches a sublinear speedup.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
