基于删失、量化和广义群ADMM的通信高效分布式学习
Communication Efficient Distributed Learning with Censored, Quantized, and Generalized Group ADMM
摘要：
在本文中，我们提出了一个有效的通信分散的机器学习框架，解决了一个共识优化问题定义在网络上的相互连接的工人。所提出的算法，删失和量化广义GADMM（CQ-GGADMM），利用工人分组和分散学习的思想组交替方向乘法（GADMM），并通过扩展其适用性广义网络拓扑结构，推动通信效率的前沿，同时将链接删失量化后可忽略不计的更新。我们从理论上证明了在一些温和的假设下，当局部目标函数是强凸的时，CQ-GGADMM具有线性收敛速度。数值模拟证实，CQ-GGADMM具有更高的通信效率，在通信轮数和传输能量消耗方面，而不影响精度和收敛速度，相比，审查分散ADMM，和工作分组方法的GADMM。
英文摘要：
In this paper, we propose a communication-efficiently decentralized machine learning framework that solves a consensus optimization problem defined over a network of inter-connected workers. The proposed algorithm, Censored and Quantized Generalized GADMM (CQ-GGADMM), leverages the worker grouping and decentralized learning ideas of Group Alternating Direction Method of Multipliers (GADMM), and pushes the frontier in communication efficiency by extending its applicability to generalized network topologies, while incorporating link censoring for negligible updates after quantization. We theoretically prove that CQ-GGADMM achieves the linear convergence rate when the local objective functions are strongly convex under some mild assumptions. Numerical simulations corroborate that CQ-GGADMM exhibits higher communication efficiency in terms of the number of communication rounds and transmit energy consumption without compromising the accuracy and convergence speed, compared to the censored decentralized ADMM, and the worker grouping method of GADMM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
