arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2005.05158math.OCstat.ML
收藏

具有增广拉格朗日量和邻近步长的不精确随机广义条件梯度

Inexact and Stochastic Generalized Conditional Gradient with Augmented Lagrangian and Proximal Step

Antonio Silveti-Falls, Cesare Molinari, Jalal Fadili

摘要:

在本文中,我们提出并分析了不精确和随机版本的CGALP算法在作者的前一篇论文中,我们表示ICGALP,允许在几个重要的数量的计算错误。特别地,这允许人们以不精确的方式计算一些梯度、近似项和/或线性最小化预言,这有助于将算法实际应用于计算密集型设置,例如在机器学习问题中常见的高维(或可能无限)希尔伯特空间中。该算法能够解决复合极小化问题,涉及三个凸真下连续函数的和受到形式为$Ax=b$的仿射约束的一些有界线性算子$A$。目标函数中只有一个函数是可微的,另外两个函数是可达的prox算子和线性最小化预言。作为主要结果,我们显示收敛的拉格朗日的最佳和渐近可行的仿射约束,以及弱收敛的对偶变量的对偶问题的解决方案,所有的几乎必然的意义。对于拉格朗日值和可行性间隙,给出了逐点和遍历的几乎必然收敛率。数值实验验证了预测的收敛速度。

英文摘要:

In this paper we propose and analyze inexact and stochastic versions of the CGALP algorithm developed in the authors' previous paper, which we denote ICGALP, that allows for errors in the computation of several important quantities. In particular this allows one to compute some gradients, proximal terms, and/or linear minimization oracles in an inexact fashion that facilitates the practical application of the algorithm to computationally intensive settings, e.g. in high (or possibly infinite) dimensional Hilbert spaces commonly found in machine learning problems. The algorithm is able to solve composite minimization problems involving the sum of three convex proper lower-semicontinuous functions subject to an affine constraint of the form $Ax=b$ for some bounded linear operator $A$. Only one of the functions in the objective is assumed to be differentiable, the other two are assumed to have an accessible prox operator and a linear minimization oracle. As main results, we show convergence of the Lagrangian to an optimum and asymptotic feasibility of the affine constraint as well as weak convergence of the dual variable to a solution of the dual problem, all in an almost sure sense. Almost sure convergence rates, both pointwise and ergodic, are given for the Lagrangian values and the feasibility gap. Numerical experiments verifying the predicted rates of convergence are shown as well.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑