The Proximal Method of Multipliers for a Class of Nonsmooth Convex Optimization

贴近非线性共轭梯度法用于复合优化

Proximal Nonlinear Conjugate Gradient Methods for Composite Optimization

本文提出一种贴近非线性共轭梯度法，用于解决复合优化问题，即光滑非凸函数与非光滑凸函数之和。方法利用前向-后向残差和三阶Hestenes-Stiefel公式，证明了在凸和弱凸情况下算法的全局收敛性，并在非凸情况下展示了更好的性能。

一种求解凸变分逆问题的近端分解方法

A Proximal Decomposition Method for Solving Convex Variational Inverse Problems

提出一种用于Hilbert空间中多个凸函数和最小化问题的近端分解算法，可处理任意数量的非光滑函数，并证明其收敛性，在信号与图像处理中展示数值应用。

用于凸优化的内在黎曼近端梯度法

The Intrinsic Riemannian Proximal Gradient Method for Convex Optimization

研究哈达玛流形上的凸优化问题，提出内在凸黎曼近端梯度法，该方法在非光滑步骤使用流形近端映射，无需在嵌入或切空间操作，建立了收敛速率并推导不等式，数值实验显示其相对于现有方法有计算优势。

A New Preconditioning Approach for an Interior Point-Proximal Method of Multipliers for Linear and Convex Quadratic Programming

In this paper, we address the efficient numerical solution of linear and quadratic programming problems, often of large scale. With this aim, we devise an infeasible interior point method, blended with the proximal method of multipliers, which in turn results in a primal-dual regularized interior point method. Application of this method gives rise to a sequence of increasingly ill-conditioned linear systems which cannot always be solved by factorization methods, due to memory and CPU time restrictions. We propose a novel preconditioning strategy which is based on a suitable sparsification of t…

A Levenberg-Marquardt Method for Nonsmooth Regularized Least Squares

We develop a Levenberg-Marquardt method for minimizing the sum of a smooth nonlinear least-squar es term $f(x) = \tfrac{1}{2} \|F(x)\|_2^2$ and a nonsmooth term $h$. Both $f$ and $h$ may be nonconvex. Steps are computed by minimizing the sum of a regularized linear least-squares model and a model of $h$ using a first-order method such as the proximal gradient method. We establish global convergence to a first-order stationary point of both a trust-region and a regularization variant of the Levenberg-Marquardt method under the assumptions that $F$ and its Jacobian are Lipschitz continuous and $…

快速近端线性化交替方向乘子法与并行分裂

Fast Proximal Linearized Alternating Direction Method of Multiplier with Parallel Splitting

本文提出快速近端增广拉格朗日法和快速近端ADMM并行分裂法，改进了收敛速度并降低了计算复杂度，实验证明其在合成和真实数据上均优于传统PALM和ADMM。

Global Convergence of Model Function Based Bregman Proximal Minimization Algorithms

Lipschitz continuity of the gradient mapping of a continuously differentiable function plays a crucial role in designing various optimization algorithms. However, many functions arising in practical applications such as low rank matrix factorization or deep neural network problems do not have a Lipschitz continuous gradient. This led to the development of a generalized notion known as the $L$-smad property, which is based on generalized proximity measures called Bregman distances. However, the $L$-smad property cannot handle nonsmooth functions, for example, simple nonsmooth functions like $\a…

非光滑优化中基于变量度量的近似步长方法

Variable metric inexact line-search based methods for nonsmooth optimization

本文提出一种新的近似梯度方法，用于最小化可微非凸函数与凸函数之和。通过定义合适的下降方向和Armijo规则，证明了非凸情况下的极限点为 stationary 点，凸情况下序列收敛于极小值，且在梯度Lipschitz条件下具有O(1/k)收敛速度。

Stochastic Proximal Gradient Algorithm with Minibatches. Application to Large Scale Learning Models

Stochastic optimization lies at the core of most statistical learning models. The recent great development of stochastic algorithmic tools focused significantly onto proximal gradient iterations, in order to find an efficient approach for nonsmooth (composite) population risk functions. The complexity of finding optimal predictors by minimizing regularized risk is largely understood for simple regularizations such as $\ell_1/\ell_2$ norms. However, more complex properties desired for the predictor necessitates highly difficult regularizers as used in grouped lasso or graph trend filtering. In …

Distributed Proximal Splitting Algorithms with Rates and Acceleration

We analyze several generic proximal splitting algorithms well suited for large-scale convex nonsmooth optimization. We derive sublinear and linear convergence results with new rates on the function value suboptimality or distance to the solution, as well as new accelerated versions, using varying stepsizes. In addition, we propose distributed variants of these algorithms, which can be accelerated as well. While most existing results are ergodic, our nonergodic results significantly broaden our understanding of primal-dual optimization algorithms.