arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2108.05974cs.LG
收藏

联邦学习的算子分裂视角

An Operator Splitting View of Federated Learning

University of Waterloo(滑铁卢大学);Huawei Noah’s Ark Lab(华为诺亚实验室)

Saber Malekmohammadi, Kiarash Shaloudegi, Zeou Hu, Yaoliang Yu

AI总结:

本文从算子分裂视角统一理解联邦学习算法,便于比较、改进收敛结果并发现新变体,揭示步长关键作用,且无需通信开销即可加速算法,实验验证于凸与非凸模型。

摘要:

在过去几年中,联邦学习($\ exttt{FL}$)社区见证了新$\ exttt{FL}$算法的激增。然而,我们对$\ exttt{FL}$理论的理解仍然支离破碎,对这些算法进行彻底、正式的比较仍然难以实现。受这一差距的驱动,我们表明许多现有的$\ exttt{FL}$算法可以从算子分裂的角度来理解。这种统一使我们能够轻松比较不同的算法,改进先前的收敛结果,并发现新的算法变体。特别是,我们的分析揭示了步长在$\ exttt{FL}$算法中发挥的关键作用。这种统一还导致了一种简化和经济的方式来加速$\ exttt{FL}$算法,而不会产生任何通信开销。我们在凸和非凸模型上进行了数值实验,以验证我们的发现。

英文摘要:

Over the past few years, the federated learning ($\texttt{FL}$) community has witnessed a proliferation of new $\texttt{FL}$ algorithms. However, our understating of the theory of $\texttt{FL}$ is still fragmented, and a thorough, formal comparison of these algorithms remains elusive. Motivated by this gap, we show that many of the existing $\texttt{FL}$ algorithms can be understood from an operator splitting point of view. This unification allows us to compare different algorithms with ease, to refine previous convergence results and to uncover new algorithmic variants. In particular, our analysis reveals the vital role played by the step size in $\texttt{FL}$ algorithms. The unification also leads to a streamlined and economic way to accelerate $\texttt{FL}$ algorithms, without incurring any communication overhead. We perform numerical experiments on both convex and nonconvex models to validate our findings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑