FedSEA: 在联邦在线学习中实现并行化的优势
FedSEA: Achieving Benefit of Parallelization in Federated Online Learning
- CMInDS, IIT Bombay, India（CMInDS，印度班加罗尔理工学院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文提出FedSEA算法，通过引入随机扩展对手模型，解决了联邦在线学习中并行化优势的问题，为平滑和强凸损失函数分别证明了O(√T)和O(log T)的全局网络遗憾界。
AI中文摘要：
在线联邦学习（OFL）已成为一种流行的框架，用于在连续数据流上进行去中心化决策，同时不牺牲客户端隐私。然而，标准OFL中假设的对手模型通常排除了任何潜在的并行化优势。此外，它未能充分捕捉OFL问题中的不同统计变化来源。在本文中，我们通过整合随机扩展对手（SEA）扩展OFL范式。在此框架下，损失函数在客户端之间随时间保持固定。然而，对手在每个时间点独立地为每个客户端选择数据分布。我们提出算法OFL来解决这个问题，该算法利用客户端上的在线随机梯度下降，以及通过服务器进行的定期全局聚合。我们建立了时间跨度T内的全局网络遗憾界，针对两类函数：（1）对于平滑且凸损失函数，我们证明了O(√T)的界；（2）对于平滑且强凸损失函数，我们证明了O(log T)的界。通过仔细分析，我们量化了空间（跨客户端）和时间（随时间）数据异质性对遗憾界的影响。因此，我们识别出一个温和的时间变化（相对于随机梯度方差）的领域，其中网络遗憾随着并行化而改善。因此，在SEA设置中，我们的结果改进了在线联邦学习中现有的悲观最坏情况结果。
英文摘要：
Online federated learning (OFL) has emerged as a popular framework for decentralized decision-making over continuous data streams without compromising client privacy. However, the adversary model assumed in standard OFL typically precludes any potential benefits of parallelization. Further, it fails to adequately capture the different sources of statistical variation in OFL problems. In this paper, we extend the OFL paradigm by integrating a stochastically extended adversary (SEA). Under this framework, the loss function remains fixed across clients over time. However, the adversary dynamically and independently selects the data distribution for each client at each time. We propose the \algoOFL{} algorithm to solve this problem, which utilizes online stochastic gradient descent at the clients, along with periodic global aggregation via the server. We establish bounds on the global network regret over a time horizon \(T\) for two classes of functions: (1) for smooth and convex losses, we prove an \(\mathcal{O}(\sqrt{T})\) bound, and (2) for smooth and strongly convex losses, we prove an \(\mathcal{O}(\log T)\) bound. Through careful analysis, we quantify the individual impact of both spatial (across clients) and temporal (over time) data heterogeneity on the regret bounds. Consequently, we identify a regime of mild temporal variation (relative to stochastic gradient variance), where the network regret improves with parallelization. Hence, in the SEA setting, our results improve the existing pessimistic worst-case results in online federated learning.
