方差缩减是拜占庭式的解药：更好的利率，更弱的假设和通信压缩作为顶部的樱桃
Variance Reduction is an Antidote to Byzantines: Better Rates, Weaker Assumptions and Communication Compression as a Cherry on the Top
摘要：
由于协作学习和联邦学习的兴趣日益增长，拜占庭鲁棒性已经获得了很多关注。然而，许多富有成效的方向，如使用方差减少实现鲁棒性和通信压缩，以减少通信成本，仍然在该领域的薄弱探索。这项工作解决了这个差距，并提出了Byz-VR-MARINA -一个新的拜占庭宽容的方法，减少方差和压缩。我们的论文的一个关键信息是，方差减少是更有效地打击拜占庭工人的关键。同时，通信压缩是一个额外的好处，使该过程更有效的通信。我们得到理论上的收敛保证Byz-VR-MARINA优于以前的国家的最先进的一般非凸和Polyak-Lojasiewicz损失函数。不像并发拜占庭强大的方法与方差减少和/或压缩，我们的复杂性的结果是紧密的，不依赖于限制性的假设，如有界的梯度或有限的压缩。此外，我们提供了第一次分析的拜占庭宽容的方法，支持非均匀采样的随机梯度。数值实验证实了我们的理论研究结果。
英文摘要：
Byzantine-robustness has been gaining a lot of attention due to the growth of the interest in collaborative and federated learning. However, many fruitful directions, such as the usage of variance reduction for achieving robustness and communication compression for reducing communication costs, remain weakly explored in the field. This work addresses this gap and proposes Byz-VR-MARINA - a new Byzantine-tolerant method with variance reduction and compression. A key message of our paper is that variance reduction is key to fighting Byzantine workers more effectively. At the same time, communication compression is a bonus that makes the process more communication efficient. We derive theoretical convergence guarantees for Byz-VR-MARINA outperforming previous state-of-the-art for general non-convex and Polyak-Lojasiewicz loss functions. Unlike the concurrent Byzantine-robust methods with variance reduction and/or compression, our complexity results are tight and do not rely on restrictive assumptions such as boundedness of the gradients or limited compression. Moreover, we provide the first analysis of a Byzantine-tolerant method supporting non-uniform sampling of stochastic gradients. Numerical experiments corroborate our theoretical findings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
