异步COMID：参数服务器上传输数据稀疏化技巧的理论基础
Asynchronous COMID: the theoretic basis for transmitted data sparsification tricks on Parameter Server
摘要：
异步FTRL-proximal和L2范数在服务器端完成是参数服务器中广泛使用的两个技巧，参数服务器是延迟SGD的实现。它们的共同点是将部分更新计算留在服务器上，通过使传输的数据稀疏化来减轻网络的负担。但上述技巧的收敛性并没有得到很好的证明。基于上述共性，本文提出了一种更为通用的异步COMID算法，并证明了其收敛性。我们证明了异步FTRL-proximal和L2范数是异步COMID的应用，证明了上述两个技巧的收敛性。然后，我们进行实验来验证理论结果。实验结果表明，与参数服务器上的延时SGD相比，异步COMID在不影响数学收敛速度和最终输出的情况下，减轻了网络负担。
英文摘要：
Asynchronous FTRL-proximal and L2 norm done at server are two widely used tricks in Parameters Server which is an implement of delayed SGD. Their commonness is leaving parts of updating computation on server which reduces the burden of network via making transmitted data sparse. But above tricks' convergences are not well-proved. In this paper, based on above commonness, we propose a more general algorithm named as asynchronous COMID and prove its convergence. We prove that asynchronous FTRL-proximal and L2 norm done at server are applications of asynchronous COMID, which demonstrates the convergences of above two tricks. Then, we conduct experiments to verify theoretical results. Experimental results show that compared with delayed SGD on Parameters Server, asynchronous COMID reduces the burden of the network without any harm on the mathematical convergence speed and final output.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
