V. L. Sousa Junior
In this paper, we study the convergence of an interior subgradient and proximal methods for a DC (difference of convex functions) constrained minimization problem.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
We present two approximate versions of the proximal subgradient method for minimizing the sum of two convex functions (not necessarily differentiable). The algorithms involve, at each iteration, inexact evaluations of the proximal operator and approximate subgradients of the functions (namely: the $ε$-subgradients). The methods use different error criteria for approximating the proximal operators. We provide an analysis of the convergence and rate of convergence properties of these methods, considering various stepsize rules, including both, diminishing and constant stepsizes. For the case whe…
In this paper we present a variant of the proximal forward-backward splitting iteration for solving nonsmooth optimization problems in Hilbert spaces, when the objective function is the sum of two nondifferentiable convex functions. The proposed iteration, which will be called Proximal Subgradient Splitting Method, extends the classical subgradient iteration for important classes of problems, exploiting the additive structure of the objective function. The weak convergence of the generated sequence was established using different stepsizes and under suitable assumptions. Moreover, we analyze t…
Locating proximal points is a component of numerous minimization algorithms. This work focuses on developing a method to find the proximal point of a convex function at a point, given an inexact oracle. Our method assumes that exact function values are at hand, but exact subgradients are either not available or not useful. We use approximate subgradients to build a model of the objective function, and prove that the method converges to the true prox-point within acceptable tolerance. The subgradient $g_k$ used at each step $k$ is such that the distance from $g_k$ to the true subdifferential of…
We analyze the stochastic proximal subgradient descent in the case where the objective functions are path differentiable and verify a Sard-type condition. While the accumulation set may not be reduced to unique point, we show that the time spent by the iterates to move from one accumulation point to another goes to infinity. An oscillation-type behavior of the drift is established. These results show a strong stability property of the proximal subgradient descent. Using the theory of closed measures, Bolte, Pauwels and Ríos-Zertuche established this type of behavior for the deterministic subgr…
Proximal point algorithm has found many applications, and it has been playing fundamental roles in the understanding, design, and analysis of many first-order methods. In this paper, we derive the tight convergence rate in subgradient norm of the proximal point algorithm, which was conjectured by Taylor, Hendrickx and Glineur [SIAM J.~Optim., 27 (2017), pp.~1283--1313]. This sort of convergence results in terms of the residual (sub)gradient norm is particularly interesting when considering dual methods, where the dual residual gradient norm corresponds to the primal distance to feasibility.
Universal subgradient and proximal bundle methods for convex and strongly convex hybrid composite optimization
本文提出两种通用方法，用于解决凸和强凸混合复合优化问题，无需参数且无需最优值知识，具有高效的复杂度界。
The Boosted Double-Proximal Subgradient Algorithm for Nonconvex Optimization
本文提出BDSA算法解决一般结构化非光滑非凸优化问题，结合次梯度与近端步骤并集成线搜索，在Kurdyka--Lojasiewicz性质下证明收敛性并分析收敛速率，实验验证其逃离非最优临界点的能力及实际应用有效性。
Stochastic Approximation Proximal Subgradient Method for Stochastic Convex-Concave Minimax Optimization
本文针对随机凸-凹极小极大优化提出SAPS方法，并在有界次梯度条件不成立时进一步提出LSAAL方法，证明了两种算法均具有O(N^{-1/2})期望收敛速率及相应的高概率界。
A proximal subgradient algorithm for constrained multiobjective DC-type optimization
本文提出了一种近端subgradient算法，用于解决受约束的多目标DC型优化问题，通过建立最优性条件并保证序列有界性，实现了对非凸问题的有效求解。
A proximal subgradient algorithm with extrapolation for structured nonconvex nonsmooth problems
本文针对一类结构化非凸非光滑优化问题，提出一种带外推的近端次梯度算法，证明其子序列收敛及在KL性质下的全局收敛性，并通过非凸正则化最小二乘和含分布式能源的最优潮流实验验证了算法性能。
Proximal Algorithms for a class of abstract convex functions
该研究基于抽象凸性与广义次梯度定义抽象近端算子，提出三种适用于一类非凸优化问题的算法，并证明其全局收敛性。
In this paper, we consider a broad class of nonsmooth and nonconvex fractional programs, where the numerator can be written as the sum of a continuously differentiable convex function whose gradient is Lipschitz continuous and a proper lower semicontinuous (possibly nonconvex) function, and the denominator is weakly convex over the constraint set. This model problem includes the composite optimization problems studied extensively lately, and encompasses many important modern fractional optimization problems arising from diverse areas such as the recently proposed scale invariant sparse signal …
A Single-loop Proximal Subgradient Algorithm for A Class Structured Fractional Programs
针对一类含线性算子的非凸非光滑结构化分式规划，提出单循环邻近次梯度算法及非单调线搜索变体，证明其全局收敛并在三类应用中验证优越性。
In this paper, we introduce a stochastic projected subgradient method for weakly convex (i.e., uniformly prox-regular) nonsmooth, nonconvex functions---a wide class of functions which includes the additive and convex composite classes. At a high-level, the method is an inexact proximal point iteration in which the strongly convex proximal subproblems are quickly solved with a specialized stochastic projected subgradient method. The primary contribution of this paper is a simple proof that the proposed algorithm converges at the same rate as the stochastic gradient method for smooth nonconvex p…
Two distributed algorithms are described that enable all users connected over a network to cooperatively solve the problem of minimizing the sum of all users' objective functions over the intersection of all users' constraint sets, where each user has its own private nonsmooth convex objective function and closed convex constraint set, which is the intersection of a number of simple, closed convex sets. One algorithm enables each user to adjust its estimate by using a proximity operator of its objective function and the metric projection onto one set randomly selected from the simple, closed c…
An Inexact Variable Metric Proximal Gradient-subgradient Algorithm for a Class of Fractional Optimization Problems
研究一类分数优化问题，提出不精确变度量近端梯度 - 次梯度算法（iVPGSA），它能灵活近似求解子问题，适应性强且效率高。基于KL性质证明其全局收敛并刻画速率，数值实验显示该算法优于现有算法。
We prove that the proximal stochastic subgradient method, applied to a weakly convex problem, drives the gradient of the Moreau envelope to zero at the rate $O(k^{-1/4})$. As a consequence, we resolve an open question on the convergence rate of the proximal stochastic gradient method for minimizing the sum of a smooth nonconvex function and a convex proximable function.
Three Operator Splitting (TOS) (Davis & Yin, 2017) can minimize the sum of multiple convex functions effectively when an efficient gradient oracle or proximal operator is available for each term. This requirement often fails in machine learning applications: (i) instead of full gradients only stochastic gradients may be available; and (ii) instead of proximal operators, using subgradients to handle complex penalty functions may be more efficient and realistic. Motivated by these concerns, we analyze three potentially valuable extensions of TOS. The first two permit using subgradients and stoch…
A Unified Analysis on the Subgradient Upper Bounds for the Subgradient Methods Minimizing Composite Nonconvex, Nonsmooth and Non-Lipschitz Functions
本文提出了一种统一分析方法，用于研究子梯度方法在最小化复合非凸、非光滑和非利普斯芝函数时的子梯度上界问题，并扩展到随机近端子梯度算法。
The problem of minimizing the sum of nonsmooth, convex objective functions defined on a real Hilbert space over the intersection of fixed point sets of nonexpansive mappings, onto which the projections cannot be efficiently computed, is considered. The use of proximal point algorithms that use the proximity operators of the objective functions and incremental optimization techniques is proposed for solving the problem. With the focus on fixed point approximation techniques, two algorithms are devised for solving the problem. One blends an incremental subgradient method, which is a useful algor…
We study the time optimal control problem for differential inclusions with a general closed target. We first give the representation of the proximal horizontal subgradients of the minimum time function $\mathcal{T}$ and then, together with the representation of the proximal subgradients, we obtain some relationships between the normal cones to the sublevel of $\mathcal{T}$ and the normal cones to its epigraph. The relationships allow us to get the propagation of the proximal subdifferential as well as of the proximal horizontal subdifferential of $\mathcal{T}$ along optimal trajectories. Final…
In this short survey, I revisit the role of the proximal point method in large scale optimization. I focus on three recent examples: a proximally guided subgradient method for weakly convex stochastic approximation, the prox-linear algorithm for minimizing compositions of convex functions and smooth maps, and Catalyst generic acceleration for regularized Empirical Risk Minimization.
In this paper, we consider a proximal linearized alternating direction method of multipliers (PL-ADMM) for solving linearly constrained nonconvex and possibly nonsmooth optimization problems. The algorithm is generalized by using variable metric proximal terms in the primal updates and an over-relaxation stepsize in the multiplier update. We prove that the sequence generated by this method is bounded and its limit points are critical points. Under the powerful Kurdyka-{Łojasiewicz} properties we prove that the sequence has a finite length thus converges, and we drive its convergence rates.
A nonmonotone extrapolated proximal gradient-subgradient algorithm beyond global Lipschitz gradient continuity
本文提出一种无需问题参数的新型算法，通过精心设计的非单调线搜索和外推步骤，解决非全局Lipschitz梯度连续性问题，并在Kurdyka-Ł ojasiewicz性质下建立收敛分析。
Incremental Gradient, Subgradient, and Proximal Methods for Convex Optimization: A Survey
本文综述了用于最小化大量凸函数和的递增方法，提出了一种统一框架，涵盖梯度、次梯度及新近端方法，分析了收敛性和速率，并讨论了在推断/机器学习、信号处理和大规模分布式优化中的应用。
"Classical" First Order (FO) algorithms of convex optimization, such as Mirror Descent algorithm or Nesterov's optimal algorithm of smooth convex optimization, are well known to have optimal (theoretical) complexity estimates which do not depend on the problem dimension. However, to attain the optimality, the domain of the problem should admit a "good proximal setup". The latter essentially means that 1) the problem domain should satisfy certain geometric conditions of "favorable geometry", and 2) the practical use of these methods is conditioned by our ability to compute at a moderate cost {\…
For first-order smooth optimization, the research on the acceleration phenomenon has a long-time history. Until recently, the mechanism leading to acceleration was not successfully uncovered by the gradient correction term and its equivalent implicit-velocity form. Furthermore, based on the high-resolution differential equation framework with the corresponding emerging techniques, phase-space representation and Lyapunov function, the squared gradient norm of Nesterov's accelerated gradient descent (\texttt{NAG}) method at an inverse cubic rate is discovered. However, this result cannot be dire…
We develop model-based methods for solving stochastic convex optimization problems, introducing the approximate-proximal point, or aProx, family, which includes stochastic subgradient, proximal point, and bundle methods. When the modeling approaches we propose are appropriately accurate, the methods enjoy stronger convergence and robustness guarantees than classical approaches, even though the model-based methods typically add little to no computational overhead over stochastic subgradient methods. For example, we show that improved models converge with probability 1 and enjoy optimal asymptot…
We develop subgradient- and gradient-based methods for minimizing strongly convex functions under a notion which generalizes the standard Euclidean strong convexity. We propose a unifying framework for subgradient methods which yields two kinds of methods, namely, the Proximal Gradient Method (PGM) and the Conditional Gradient Method (CGM), unifying several existing methods. The unifying framework provides tools to analyze the convergence of PGMs and CGMs for non-smooth, (weakly) smooth, and further for structured problems such as the inexact oracle models. The proposed subgradient methods yie…
In this paper we propose a proximal algorithm for minimizing an objective function of two block variables consisting of three terms: 1) a smooth function, 2) a nonsmooth function which is a composition between a strictly increasing, concave, differentiable function and a convex nonsmooth function, and 3) a smooth function which couples the two block variables. We propose a variable metric composite proximal alternating linearized minimization (CPALM) to solve this class of problems. Building on the powerful Kurdyka-Łojasiewicz property, we derive the convergence analysis and establish that eac…
In this paper we study nonconvex and nonsmooth optimization problems with semi-algebraic data, where the variables vector is split into several blocks of variables. The problem consists of one smooth function of the entire variables vector and the sum of nonsmooth functions for each block separately. We analyze an inertial version of the Proximal Alternating Linearized Minimization (PALM) algorithm and prove its global convergence to a critical point of the objective function at hand. We illustrate our theoretical findings by presenting numerical experiments on blind image deconvolution, on sp…
Understanding the role that subgradients play in various second-order variational analysis constructions can help us uncover new properties of important classes of functions in variational analysis. Focusing mainly on the behavior of the second subderivative and subgradient proto-derivative of polyhedral functions, functions with polyhedral epigraphs, we demonstrate that choosing the underlying subgradient, utilized in the definitions of these concepts, from the relative interior of the subdifferential of polyhedral functions ensures stronger second-order variational properties such as strict …
Two-step inertial Bregman proximal alternating linearized minimization algorithm for nonconvex and nonsmooth problems
针对非凸非光滑不可分优化问题，本文结合两步惯性外推与Bregman距离，基于PALM提出新型迭代算法，证明序列收敛性并通过信号恢复等问题验证有效性。
A proximal-linearized NEP method for composite optimization with conic and manifold constraints
本文提出一种非精确近端线性化精确惩罚算法，用于求解带锥约束与流形约束的DC复合优化问题，在温和假设下达到O(ε^{-2})复杂度，数值实验验证其有效性。
A proximal subgradient method for nonconvex stochastic optimization under the Kurdyka-Łojasiewicz condition
本研究提出满足Kurdyka-Łojasiewicz条件的邻近随机次梯度方法，可处理更通用非凸随机优化问题，保证轨迹收敛性并导出多项式收敛速率。
A stochastic two-step inertial Bregman proximal alternating linearized minimization algorithm for nonconvex and nonsmooth problems
本文提出STiBPALM算法，结合两步惯性与Bregman近端交替线性化及方差缩减随机梯度，用于求解大规模非凸非光滑优化问题，并证明其收敛性，通过稀疏非负矩阵分解和盲图像去模糊实验验证性能。
In this paper, we propose a sparse equity portfolio optimization (SEPO) based on the mean-variance portfolio selection model. Aimed at minimizing transaction cost by avoiding small investments, this new model includes $\ell_0$-norm regularization of the asset weights to promote sparsity, hence the acronym SEPO-$\ell_0$. The selection model is also subjected to a minimum expected return. The complexity of the model calls for proximal method, which allows us to handle the objectives terms separately via the corresponding proximal operators. We develop an efficient ADMM-like algorithm to find the…
A Decentralized Proximal Gradient Tracking Algorithm for Composite Optimization on Riemannian Manifolds
针对黎曼子流形上的复合优化问题，提出去中心化近端梯度跟踪算法，充分利用复合结构，将收敛迭代复杂度降至$\mathcal{O}(ε^{-2})$，数值实验验证了其高效性。
Mini-batch stochastic subgradient for functional constrained optimization
针对含多函数约束的有限和复合凸优化问题，提出将其重构为随机优化问题的小批量随机次梯度投影算法，推导步长表达式与切换规则，证明次线性收敛速率且数值性能优于单批量方案。
Stochastic Bregman Subgradient Methods for Nonsmooth Nonconvex Optimization Problems
本文研究目标函数缺乏Clarke正则性时的随机Bregman次梯度方法，通过微分包含建立收敛性，提出非精确求解与动量集成机制，并验证其在训练非光滑神经网络中的有效性。
A Bregman proximal linearized ADMM for fractional programming with nonlinear coupling constraints
本文针对带非线性耦合约束的非凸分式规划问题，提出Bregman邻近线性化ADMM，证明其收敛性与收敛速率，数值实验验证了算法有效性。
In this paper, we first study nonsmooth steepest descent method for nonsmooth functions defined on Hilbert space and establish the corresponding algorithm by proximal subgradients. Then, we use this algorithm to find stationary points for those functions satisfying prox-regularity and Lipschitzian continuity. As one application, the established algorithm is used to search the minimizer of lower semicontinuous and convex functions on finite-dimensional space. The convergent theorem, as one extension and improvement of the existing converging result for twice continuously differentiable convex f…
Smoothness of Subgradient Mappings and Its Applications in Parametric Optimization
本文研究邻近正则函数次梯度映射的严格原可微性，证明其可替代梯度光滑性，并以相对内部条件刻画 C2可分解函数邻近映射的连续可微性及 KKT 系统正则性。
A proximal algorithm incorporating difference of convex functions optimization for solving a class of single-ratio fractional programming
针对一类可由球面非光滑问题转化而来的单比分式最小化问题，推导最优性条件，提出结合凸函数差优化的PS-DCA算法，证明聚点为临界点及典型问题全局收敛，实验显示其对初值不敏感、不易陷入低质量局部极小值。
This paper is devoted to the study of second order optimality conditions for strong local minimizers in the frameworks of unconstrained and constrained optimization problems in finite dimensions via subgradient graphical derivative. We prove that the positive definiteness of the subgradient graphical derivative of an extended-real-valued lower semicontinuous proper function at a proximal stationary point is sufficient for the quadratic growth condition. It is also a necessary condition for the latter property when the function is either subdifferentially continuous, prox-regular, twice epi-dif…
This work considers the question: what convergence guarantees does the stochastic subgradient method have in the absence of smoothness and convexity? We prove that the stochastic subgradient method, on any semialgebraic locally Lipschitz function, produces limit points that are all first-order stationary. More generally, our result applies to any function with a Whitney stratifiable graph. In particular, this work endows the stochastic subgradient method, and its proximal extension, with rigorous convergence guarantees for a wide class of problems arising in data science---including all popula…
In this paper, we first propose a general inertial proximal point method for the mixed variational inequality (VI) problem. Based on our knowledge, without stronger assumptions, convergence rate result is not known in the literature for inertial type proximal point methods. Under certain conditions, we are able to establish the global convergence and a $o(1/k)$ convergence rate result (under certain measure) of the proposed general inertial proximal point method. We then show that the linearized alternating direction method of multipliers (ADMM) for separable convex optimization with linear co…
The Convergence Properties of Infeasible Inexact Proximal Alternating Linearized Minimization
本文研究了不可行非精确近端交替线性化极小化方法（PALM-I）的收敛性质，通过构造代理序列克服目标值非单调性问题并设计可实现的非精确准则，证明了聚点稳定性、迭代收敛性及渐近收敛速率，数值实验验证了其在CPU时间上的优势。
Dynamical convergence analysis for nonconvex linearized proximal ADMM algorithms
本文利用连续时间动力系统分析非凸复合优化的线性化近端 ADMM 算法，借助 KL 性质证明了确定与随机变体的全局收敛性及收敛速率，并提出加速版本。
This paper proposes and develops a new Newton-type algorithm to solve subdifferential inclusions defined by subgradients of extended-real-valued prox-regular functions. The proposed algorithm is formulated in terms of the second-order subdifferential of such functions that enjoys extensive calculus rules and can be efficiently computed for broad classes of extended-real-valued functions. Based on this and on metric regularity and subregularity properties of subgradient mappings, we establish verifiable conditions ensuring well-posedness of the proposed algorithm and its local superlinear conve…
The Güler-type acceleration for proximal gradient, linearized augmented Lagrangian and linearized alternating direction method of multipliers
本文提出三种基于Güler加速技术的优化算法，提升收敛效率并应用于不同问题实例。
An Inexact Proximal Linearized DC Algorithm with Provably Terminating Inner Loop
针对现有不精确DC算法内循环无法保证有限终止的问题，本文利用ε-次微分提出了不精确近端线性化DC算法tPLDCA，在保证收敛性的同时实现了内循环有限次终止，并给出了可行实现与数值验证。
Bregman Proximal Linearized ADMM for Minimizing Separable Sums Coupled by a Difference of Functions
本文提出一种结合Bregman距离的线性化ADMM分裂算法，用于求解由函数差耦合的非凸非光滑可分离复合优化问题，并证明了其子序列及全序列收敛性，在鲁棒PCA和最优潮流问题上验证了有效性。
Subgame Perfect Methods in Nonsmooth Convex Optimization
本文针对带次梯度或近端算子预言机的非光滑凸优化问题，识别并提出满足子博弈完美性的算法，证明其性能不劣于极小极大最优且常更优。
A Proximal Gradient Framework for Composite Multiobjective Optimization on Riemannian Manifolds
本文提出了一种用于流形上复合优化问题的Riemannian多目标近似梯度方法（RMPGM），直接处理向量值目标并保证全局收敛到Pareto stationary点，具有O(1/k)收敛速度。进一步开发了两种变体以提高实用性和性能：允许在子问题求解中控制近似程度的不精确RMPGM，以及自适应调整惩罚参数的信赖域RMPGM，达到O(ε^{-2})迭代复杂度。数值实验表明，所提方法在性能上优于基于次梯度的基线方法。
We introduce SPRING, a novel stochastic proximal alternating linearized minimization algorithm for solving a class of non-smooth and non-convex optimization problems. Large-scale imaging problems are becoming increasingly prevalent due to advances in data acquisition and computational capabilities. Motivated by the success of stochastic optimization methods, we propose a stochastic variant of proximal alternating linearized minimization (PALM) algorithm \cite{bolte2014proximal}. We provide global convergence guarantees, demonstrating that our proposed method with variance-reduced stochastic gr…
Delayed supermartingale convergence lemmas for stochastic approximation with Nesterov momentum
本文针对带 Nesterov 动量的随机逼近中延迟信息导致难以实现几乎必然收敛的问题，提出基于延迟超鞅收敛引理的一般性收敛框架，并应用于随机次梯度、邻近 Robbins-Monro 及邻近随机次梯度等方法，通过数值实验验证了理论结果。
Primal-dual proximal bundle and conditional gradient methods for convex problems
本文提出原始-对偶近端束方法求解凸非光滑复合优化与鞍点问题，建立迭代复杂度，发现与条件梯度的对偶性，并验证其优于次梯度方法。
Stochastic subgradient for composite optimization with functional constraints
本文针对带随机复合目标函数与无限约束交集的凸优化问题，提出带随机可行性更新的随机次梯度方法，证明其在不同情形下的收敛速率，数值结果验证了方法有效性。
In this paper, two distributed multi-proximal primal-dual algorithms are proposed to deal with a class of distributed nonsmooth resource allocation problems. In these problems, the global cost function is the summation of local convex and nonsmooth cost functions, each of which consists of one twice differentiable function and multiple nonsmooth functions. Communication graphs of underling multi-agent systems are directed and strongly connected but not necessarily weighted-balanced. The multi-proximal splitting is designed to deal with the difficulty caused by the unproximable property of the …
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。