发表机构
Northeastern University; Carnegie Mellon University（东北大学; 卡内基梅隆大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对实际联邦学习中客户端非静态不可用性问题，提出轻量高效的FedSWE算法，可补偿错过计算、稳定扩散全局更新，收敛性好且开销小，经实验验证其有效性。
AI 中文摘要
由于资源限制或内外不确定性，实际联邦学习系统中的客户端通常是间歇性可用的边缘设备。在高度动态的环境中，参数服务器缺乏客户端可用性的先验实时知识，这使得将传统联邦学习算法调整为对客户端可用性不确定性具有弹性变得具有挑战性。如果处理不当，复杂的客户端可用性会引入显著偏差，可能损害训练模型的性能。大多数现有工作要么未能考虑非静态客户端可用性动态，要么需要大量内存和计算开销。本文旨在开发可证明对异构且非静态随机客户端可用性具有弹性的高效联邦学习算法。我们提出了FedSWE，它采用新颖的算法结构，尽管对非静态动态未知，仍能实现：（i）补偿错过的计算；（ii）在多轮中稳定并扩散全局更新；（iii）通过隐式闲聊均匀混合本地更新。与标准FedAvg相比，FedSWE引入的额外内存和计算开销很小。我们证明FedSWE收敛到非凸目标的一个稳定点，同时在某些特殊情况下实现所需的线性加速特性。我们在真实数据集上针对多样化的客户端不可用性动态进行的数值实验证实了我们的分析。
英文摘要
Due to resource constraints or external and internal uncertainties, clients in real-world federated learning systems are often intermittently available edge devices. In highly dynamic environments, the parameter server lacks prior real-time knowledge of clients' availability, making it challenging to adapt traditional federated learning algorithms to be resilient to uncertainties in client availability. If not carefully addressed, complex client availability can introduce significant bias, potentially harming the performance of the trained model. Most prior work either fails to account for non-stationary client availability dynamics or demands significant memory and computational overhead. This paper aims to develop efficient federated learning algorithms that are provably resilient to heterogeneous and non-stationary stochastic client availability. We propose FedSWE, which admits novel algorithmic structures to (i) compensate for missed computations, (ii) stabilize and diffuse the global updates over rounds, and (iii) evenly mix the local updates through implicit gossiping, despite being agnostic to non-stationary dynamics. Compared with the standard FedAvg, FedSWE introduces light additional memory and computation overhead. We show that FedSWE converges to a stationary point of non-convex objectives while achieving the desired linear speedup property in certain special cases. We corroborate our analysis with numerical experiments over diversified client unavailability dynamics on real-world data sets.
CommentsJournal of Machine Learning Research
