发表机构
State Key Laboratory of Novel Software Technology, Nanjing University; School of Artificial Intelligence, Nanjing University; Department of Computer Science, City University of Hong Kong（南京大学计算机软件新技术全国重点实验室; 南京大学人工智能学院; 香港城市大学计算机科学系）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对不可知联邦学习通信瓶颈，提出AFL-BR与AFL-Com算法，通过加快收敛和双向压缩降低通信复杂度，实验验证了效率提升。
AI 中文摘要
不可知联邦学习（AFL）旨在寻求一个在m个异构工作节点上表现可靠的模型，但通信仍然是瓶颈。我们通过加快收敛速度来减少同步轮数，并通过压缩来降低每轮通信成本，从而提升通信效率。我们首先提出AFL-BR，它使用带KL散度的在线镜像上升和分块重启来更新工作节点上的对偶权重。在T轮更新后，它达到O((log m)^{1/4}T^{-1/8})的平稳性速率，将收敛所需同步轮数对m的依赖从多项式级降至对数级。在AFL-BR的基础上，我们通过应用带误差反馈（EF）的双向压缩开发了AFL-Com。工作节点不是压缩局部梯度，而是将EF应用于其对偶加权梯度，从而能够在时变权重下直接控制聚合压缩误差。然后，我们在一般的δ-近似压缩器下为AFL-Com建立了O((δ^{-1}+(log m)^{1/4})T^{-1/8})的平稳性速率，并将加性幂等压缩器在共享随机性（SR）下的δ依赖性从δ^{-1}改进为δ^{-1/2}。在适当的压缩级别下，AFL-Com以更低的每轮通信成本保持与AFL-BR相同的收敛速率，从而通过Top-k将总通信复杂度降低(log m)^{1/4}倍，通过Rand-k和SR降低(log m)^{1/2}倍。实验验证了我们方法在同步和通信效率上的改进。
英文摘要
Agnostic federated learning (AFL) seeks a model that performs reliably across $m$ heterogeneous workers, but communication remains a bottleneck. We improve communication efficiency by reducing the number of synchronization rounds via faster convergence and the communication cost per round via compression. We first propose AFL-BR, which updates the dual weights over workers using online mirror ascent with KL divergence and blockwise restarts. It achieves an $O((\log m)^{1/4}T^{-1/8})$ stationarity rate after $T$ update rounds, reducing the $m$-dependence of the synchronization rounds required for convergence from polynomial to logarithmic order. Building on AFL-BR, we develop AFL-Com by applying bidirectional compression with error feedback (EF). Instead of compressing local gradients, workers apply EF to their dual-weighted gradients, enabling direct control of the aggregated compression error under time-varying weights. We then establish an $O((δ^{-1}+(\log m)^{1/4})T^{-1/8})$ stationarity rate for AFL-Com under general $δ$-approximate compressors and improve the $δ$-dependence from $δ^{-1}$ to $δ^{-1/2}$ for additive-and-idempotent compressors with shared randomness (SR). With suitable compression levels, AFL-Com retains the same convergence rate as AFL-BR at a lower per-round communication cost, yielding reductions in total communication complexity by factors of $(\log m)^{1/4}$ with Top-$k$ and $(\log m)^{1/2}$ with Rand-$k$ and SR. Experiments validate the improved synchronization and communication efficiency of our methods.
Comments30 pages, 3 figures
