arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1606.09155math.OCcs.NAmath.NAstat.ML
收藏

加速的首次-order 原始-对偶近似方法用于线性约束的复合凸编程

Accelerated first-order primal-dual proximal methods for linearly constrained composite convex programming

University of Alabama(阿拉巴马大学)

Yangyang Xu

AI总结:

本文提出两种加速方法解决线性约束的复合凸优化问题,通过线性化和近似项简化子问题,实现O(1/t)或O(1/t²)收敛性,优于现有方法。

摘要:

受大数据应用启发,近年来首次-order 方法非常流行。然而,朴素梯度方法通常收敛缓慢。因此,许多努力用于加速各种首次-order 方法。本文提出两种加速方法解决结构化的线性约束凸编程问题,假设复合凸目标。第一种方法是加速线性化增广拉格朗日方法(LALM)。在每次更新原始变量时,它允许对可微函数和增广项进行线性化,从而简化子问题。仅假设弱凸性,我们证明LALM在参数固定时具有O(1/t)收敛性,若参数自适应则可达到O(1/t²)收敛性,其中t是总迭代次数。第二种方法是加速线性化交替方向乘子法(LADMM)。除了复合凸性外,它进一步假设目标函数的两块结构。不同于经典ADMM,我们的方法允许对目标和增广项进行线性化以使更新简单。假设一个块变量的强凸性,我们证明LADMM在自适应参数下也具有O(1/t²)收敛性。这一结果优于[Goldstein等, SIIMS'14],后者要求两个块变量的强凸性且不进行目标或增广项的线性化。数值实验在二次规划、图像去噪和支持向量机上进行。所提加速方法与非加速方法及现有加速方法进行比较。结果证明了加速的有效性,并展示了所提方法在现有方法上的优越性能。

英文摘要:

Motivated by big data applications, first-order methods have been extremely popular in recent years. However, naive gradient methods generally converge slowly. Hence, much efforts have been made to accelerate various first-order methods. This paper proposes two accelerated methods towards solving structured linearly constrained convex programming, for which we assume composite convex objective. The first method is the accelerated linearized augmented Lagrangian method (LALM). At each update to the primal variable, it allows linearization to the differentiable function and also the augmented term, and thus it enables easy subproblems. Assuming merely weak convexity, we show that LALM owns $O(1/t)$ convergence if parameters are kept fixed during all the iterations and can be accelerated to $O(1/t^2)$ if the parameters are adapted, where $t$ is the number of total iterations. The second method is the accelerated linearized alternating direction method of multipliers (LADMM). In addition to the composite convexity, it further assumes two-block structure on the objective. Different from classic ADMM, our method allows linearization to the objective and also augmented term to make the update simple. Assuming strong convexity on one block variable, we show that LADMM also enjoys $O(1/t^2)$ convergence with adaptive parameters. This result is a significant improvement over that in [Goldstein et. al, SIIMS'14], which requires strong convexity on both block variables and no linearization to the objective or augmented term. Numerical experiments are performed on quadratic programming, image denoising, and support vector machine. The proposed accelerated methods are compared to nonaccelerated ones and also existing accelerated methods. The results demonstrate the validness of acceleration and superior performance of the proposed methods over existing ones.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑