线性算子的插值条件及其在性能估计问题中的应用
Interpolation Conditions for Linear Operators and Applications to Performance Estimation Problems
AI总结:
本文推广性能估计问题框架至线性算子,给出插值条件,并据此精确分析梯度法与Chambolle-Pock方法的收敛性能,改进现有结果并发现迭代平均有益。
摘要:
性能估计问题方法使得确定优化方法的精确最坏情况性能成为可能。在本工作中,我们将此框架推广到涉及线性算子的一阶方法。此扩展需要对那些线性算子的插值条件进行显式表述。我们考虑线性算子类 $\mathcal{M}:x \mapsto Mx$,其中矩阵 $M$ 具有有界奇异值,以及线性算子类,其中 $M$ 对称且具有有界特征值。我们描述了这些类的插值条件,即给定一对列表 $\{(x_i,y_i)\}$,刻画存在一个线性算子对所有 $i$ 将 $x_i$ 映射到 $y_i$ 的充分必要条件。利用这些条件,我们首先确定了应用于复合目标 $h\circ \mathcal{M}$ 的梯度方法的精确最坏情况行为,并观察到它总是对应于 $\mathcal{M}$ 为缩放算子。然后我们研究了应用于 $f+g\circ \mathcal{M}$ 的 Chambolle-Pock 方法,并改进了现有分析以获得原始-对偶间隙的精确收敛速率的证明。此外,我们研究了该方法在 Lipschitz 凸函数上的表现,并获得了最后迭代的原始精度的数值收敛速率。我们还数值上表明,在此设置中,对迭代进行平均是有益的。
英文摘要:
The Performance Estimation Problem methodology makes it possible to determine the exact worst-case performance of an optimization method. In this work, we generalize this framework to first-order methods involving linear operators. This extension requires an explicit formulation of interpolation conditions for those linear operators. We consider the class of linear operators $\mathcal{M}:x \mapsto Mx$ where matrix $M$ has bounded singular values, and the class of linear operators where $M$ is symmetric and has bounded eigenvalues. We describe interpolation conditions for these classes, i.e. necessary and sufficient conditions that, given a list of pairs $\{(x_i,y_i)\}$, characterize the existence of a linear operator mapping $x_i$ to $y_i$ for all $i$. Using these conditions, we first identify the exact worst-case behavior of the gradient method applied to the composed objective $h\circ \mathcal{M}$, and observe that it always corresponds to $\mathcal{M}$ being a scaling operator. We then investigate the Chambolle-Pock method applied to $f+g\circ \mathcal{M}$, and improve the existing analysis to obtain a proof of the exact convergence rate of the primal-dual gap. In addition, we study how this method behaves on Lipschitz convex functions, and obtain a numerical convergence rate for the primal accuracy of the last iterate. We also show numerically that averaging iterates is beneficial in this setting.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。