发表机构
The Chinese University of Hong Kong; University of Minnesota（香港中文大学; 明尼苏达大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文研究线性可分数据下使用FedAvg训练逻辑回归模型，证明任意大步长下算法稳定且目标值以O(1/R)收敛，异构性影响渐消，并通过实验验证。
AI 中文摘要
本文重新审视了使用联邦平均（FedAvg）算法训练多项逻辑回归模型的分布式学习问题。我们聚焦于任意大步长和异构更新规则的场景，其中设备在每轮通信中可能执行不同数量的本地更新。我们证明，在线性可分数据下，FedAvg 对任意步长都是稳定的，且目标值以 O(1/R) 的速率收敛到零，其中 R 是通信轮数。我们的结果还表明，设备异构性的影响渐近消失。对于足够大的 R，目标值单调递减，并以 O(1/(R T_avg)) 为界，其中 T_avg 是每轮通信中设备本地更新步数的平均值。数值实验支持了我们的发现。
英文摘要
This paper revisits the distributed learning problem for training a multinomial logistic regression model with the Federated Averaging ($\texttt{FedAvg}$) algorithm. We concentrate on a scenario with arbitrarily large stepsizes and heterogeneous update rules where the devices may perform a different number of local updates in each round. We show that, with linearly separable data, $\texttt{FedAvg}$ is stable with any stepsizes and the objective values converge to zero at the rate of ${\cal O}(1/R)$, where $R$ is the number of communication rounds. Our result also demonstrates that the effects of device heterogeneity vanish asymptotically. For sufficiently large $R$, the objective values decrease monotonically and is bounded by ${\cal O}( 1 / (R T_{\rm avg}))$, where $T_{\rm avg}$ is the average number of local update steps per communication round across devices. Numerical experiments support our findings.
Comments11 pages, 4 figures, accepted to the 65th IEEE Conference on Decision and Control
