EF21-P 及其伙伴：具有双向压缩的分布式优化的改进理论通信复杂度
EF21-P and Friends: Improved Theoretical Communication Complexity for Distributed Optimization with Bidirectional Compression
AI总结：
针对服务器与工作节点通信不可忽略的分布式优化问题，提出支持双向压缩的新方法，通过解耦压缩方差将乘性依赖转为加性依赖，在凸和非凸问题中实现最先进通信复杂度，凸机制下匹配梯度下降，并采用有偏梯度估计器。
摘要：
在这项工作中，我们关注服务器与工作节点之间通信时间不可忽略的分布式优化问题。我们提出了支持双向压缩（既从服务器到工作节点，也从工作节点到服务器）的新方法，该方法在凸和非凸问题上享有新的最先进理论通信复杂度。我们的界限是首个能够解耦来自工作节点到服务器和服务器到工作节点压缩的方差/误差的界限，将乘性依赖转化为加性依赖。此外，在凸机制下，我们获得了首个与梯度下降的理论通信复杂度相匹配的界限。即使在这种凸机制下，我们的算法也使用有偏梯度估计器，这是非标准的，需要新的证明技术，可能具有独立的意义。最后，我们的理论结果通过适当的实验得到了证实。
英文摘要：
In this work we focus our attention on distributed optimization problems in the context where the communication time between the server and the workers is non-negligible. We obtain novel methods supporting bidirectional compression (both from the server to the workers and vice versa) that enjoy new state-of-the-art theoretical communication complexity for convex and nonconvex problems. Our bounds are the first that manage to decouple the variance/error coming from the workers-to-server and server-to-workers compression, transforming a multiplicative dependence to an additive one. Moreover, in the convex regime, we obtain the first bounds that match the theoretical communication complexity of gradient descent. Even in this convex regime, our algorithms work with biased gradient estimators, which is non-standard and requires new proof techniques that may be of independent interest. Finally, our theoretical results are corroborated through suitable experiments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
