arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2103.15993math.OC
收藏

非光滑正则Optimization的拟牛顿置信域方法

A Proximal Quasi-Newton Trust-Region Method for Nonsmooth Regularized Optimization

Aleksandr Y. Aravkin, Robert Baraldi, Dominique Orban

摘要:

我们提出了一个置信域方法来最小化光滑项$f$和非光滑项$h$之和,这两个项都可以是非凸的。我们的方法的每一次迭代最小化一个可能的非凸模型的$f + h$的信任区域。该模型与$f + h$在数值和次微分上一致。当$f$满足一个光滑条件,特别是当它具有Lipschitz连续梯度,并且$h$是适当的下半连续时,我们建立了整体收敛到一阶稳定点的条件.要求$h$的模型是适当的、下半连续的和有界的。在这些弱的假设下,我们建立了一个最坏情况下的$O(1/ε^2)$迭代复杂性界匹配的最佳已知的复杂性界的标准置信域光滑优化方法。我们详细介绍了一个特殊的情况下,命名为TR-PG,在其中我们使用有限记忆拟牛顿模型的$f$和计算一个步骤与邻近梯度法,从而产生一个实用的邻近拟牛顿法。我们建立了类似的收敛性和复杂性界的二次正则化变量,命名为R2,并提供了一个解释为近端梯度法与自适应步长的非凸问题。R2也可以用于计算置信域方法内部的步骤,从而产生名为TR-R2的实现。我们描述了我们的Julia实现和报告的数值结果从稀疏优化和信号处理的反问题。TR-PG和TR-R2都表现出良好的性能,并与两种基于凸模型的线性逼近拟牛顿法相比较。

英文摘要:

We develop a trust-region method for minimizing the sum of a smooth term $f$ and a nonsmooth term $h$), both of which can be nonconvex. Each iteration of our method minimizes a possibly nonconvex model of $f + h$ in a trust region. The model coincides with $f + h$ in value and subdifferential at the center. We establish global convergence to a first-order stationary point when $f$ satisfies a smoothness condition that holds, in particular, when it has Lipschitz-continuous gradient, and $h$ is proper and lower semi-continuous. The model of $h$ is required to be proper, lower-semi-continuous and prox-bounded. Under these weak assumptions, we establish a worst-case $O(1/ε^2)$ iteration complexity bound that matches the best known complexity bound of standard trust-region methods for smooth optimization. We detail a special instance, named TR-PG, in which we use a limited-memory quasi-Newton model of $f$ and compute a step with the proximal gradient method, resulting in a practical proximal quasi-Newton method. We establish similar convergence properties and complexity bound for a quadratic regularization variant, named R2, and provide an interpretation as a proximal gradient method with adaptive step size for nonconvex problems. R2 may also be used to compute steps inside the trust-region method, resulting in an implementation named TR-R2. We describe our Julia implementations and report numerical results on inverse problems from sparse optimization and signal processing. Both TR-PG and TR-R2 exhibit promising performance and compare favorably with two linesearch proximal quasi-Newton methods based on convex models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑