精确分布一阶方法的统一与推广
A Unification and Generalization of Exact Distributed First Order Methods
摘要:
最近,分布式一阶方法的发展取得了重大进展。(At至少)已经提出了从非常不同的角度设计的两种不同类型的方法,当使用恒定步长时,它们实现精确收敛和线性收敛--这是大多数现有方法无法实现的有利特征。在本文中,我们统一,推广,并提高收敛速度的精确分布的一阶方法。我们首先进行了一个新的统一分析,揭示了如何不同的现有方法比较。分析表明,方法之间的一个主要区别是如何过去的对偶梯度相关联的增广拉格朗日对偶函数的加权。然后,我们利用分析中的见解来推导出一种新的方法-具有经过调整的过去梯度加权-改进了现有方法。我们建立了所提出的广义方法的全局R-线性收敛速度下的强凸成本Lipschitz连续梯度。
英文摘要:
Recently, there has been significant progress in the development of distributed first order methods. (At least) two different types of methods, designed from very different perspectives, have been proposed that achieve both exact and linear convergence when a constant step size is used -- a favorable feature that was not achievable by most prior methods. In this paper, we unify, generalize, and improve convergence speed of these exact distributed first order methods. We first carry out a novel unifying analysis that sheds light on how the different existing methods compare. The analysis reveals that a major difference between the methods is on how a past dual gradient of an associated augmented Lagrangian dual function is weighted. We then capitalize on the insights from the analysis to derive a novel method -- with a tuned past gradient weighting -- that improves upon the existing methods. We establish for the proposed generalized method global R-linear convergence rate under strongly convex costs with Lipschitz continuous gradients.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。