Nimit Nimana, Narin Petrot
In this work we consider an iterative method for solving the quasi-convex feasibility problem. We firstly introduce the so-called star subgradient projection operator and present some useful properties. We subsequently obtain a convergence result of the considered method by using properties of the introduced nonlinear operator.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
继续探索
In this paper, we propose a new inexact version of the projected subgradient method to solve nondifferentiable constrained convex optimization problems. The method combine $ε$-subgradient method with a procedure to obtain a feasible inexact projection onto the constraint set. Asymptotic convergence results and iteration-complexity bounds for the sequence generated by the method employing the well known exogenous stepsizes, Polyak's stepsizes, and dynamic stepsizes are established.
The convex feasibility problem (CFP) is at the core of the modeling of many problems in various areas of science. Subgradient projection methods are important tools for solving the CFP because they enable the use of subgradient calculations instead of orthogonal projections onto the individual sets of the problem. Working in a real Hilbert space, we show that the sequential subgradient projection method is perturbation resilient. By this we mean that under appropriate conditions the sequence generated by the method converges weakly, and sometimes also strongly, to a point in the intersection o…
This paper deals with the convex feasibility problem, where the feasible set is given as the intersection of a (possibly infinite) number of closed convex sets. We assume that each set is specified algebraically as a convex inequality, where the associated convex function is general (possibly non-differentiable). For finding a point satisfying all the convex inequalities we design and analyze random projection algorithms using special subgradient iterations and extrapolated stepsizes. Moreover, the iterate updates are performed based on parallel random observations of several constraint compon…
In this paper, we studied the equilibrium problem where the bi-function may be quasiconvex with respect to the second variable and the feasible set is the intersection of a finite number of convex sets. We propose a projection-algorithm, where the projection can be computed independently onto each component set. The convergence of the algorithm is investigated and numerical examples for a variational inequality problem involving affine fractional operator are provided to demonstrate the behavior of the algorithm.
Sum-Rate Maximization via Convex Optimization Using Subgradient Projections Onto Nonlinear Spectral Radius Constraint Sets
本文提出了一种基于子梯度投影的凸优化方法，用于在非线性谱半径约束集上实现总速率最大化，适用于无细胞网络等实际场景。
Several algebraic and topological properties of subgradient projection operators are investigated and various examples are provided. Connections with Moreau's proximity operator are also made and acceleration schemes for subgradient projection algorihms are discussed. Finally continuity, nonexpansiveness, monotonicity, differentiability, and epi-convergence properties are investigated.
The projected subgradient method for constrained minimization repeatedly interlaces subgradient steps for the objective function with projections onto the feasible region, which is the intersection of closed and convex constraints sets, to regain feasibility. The latter poses a computational difficulty and, therefore, the projected subgradient method is applicable only when the feasible region is "simple to project onto". In contrast to this, in the superiorization methodology a feasibility-seeking algorithm leads the overall process and objective function steps are interlaced into it. This ma…
We consider constrained minimization problems and propose to replace the projection onto the entire feasible region, required in the Projected Subgradient Method (PSM), by projections onto the individual sets whose intersection forms the entire feasible region. Specifically, we propose to perform such projections onto the individual sets in an algorithmic regime of a feasibility-seeking iterative projection method. For this purpose we use the recently developed family of Dynamic String-Averaging Projection (DSAP) methods wherein iteration-index-dependent variable strings and variable weights a…
We propose a distributed algorithm, termed the Directed-Distributed Projected Subgradient (D-DPS), to solve a constrained optimization problem over a multi-agent network, where the goal of agents is to collectively minimize the sum of locally known convex functions. Each agent in the network owns only its local objective function, constrained to a commonly known convex set. We focus on the circumstance when communications between agents are described by a directed network. The D-DPS augments an additional variable for each agent, to overcome the asymmetry caused by the directed communication n…
We propose a new subgradient method for the minimization of nonsmooth convex functions over a convex set. To speed up computations we use adaptive approximate projections only requiring to move within a certain distance of the exact projections (which decreases in the course of the algorithm). In particular, the iterates in our method can be infeasible throughout the whole procedure. Nevertheless, we provide conditions which ensure convergence to an optimal feasible point under suitable assumptions. One convergence result deals with step size sequences that are fixed a priori. Two other result…
The subgradient extragradient method for solving the variational inequality (VI) problem, which is introduced by Censor et al. \cite{CGR}, replaces the second projection onto the feasible set of the VI, in the extragradient method, with a subgradient projection onto some constructible half-space. Since the method has been introduced, many authors proposed extensions and modifications with applications to various problems. In this paper, we introduce a modified subgradient extragradient method by improving the stepsize of its second step. Convergence of the proposed method is proved under stand…
Random minibatch subgradient algorithms for convex problems with functional constraints
本文研究无限约束交集的非光滑凸优化问题，提出带有随机小批量可行性更新的次梯度迭代算法，证明了在强凸目标函数与有界次梯度条件下算法的次线性收敛速率及其最优性。
We propose Directed-Distributed Projected Subgradient (D-DPS) to solve a constrained optimization problem over a multi-agent network, where the goal of agents is to collectively minimize the sum of locally known convex functions. Each agent in the network owns only its local objective function, constrained to a commonly known convex set. We focus on the circumstance when communications between agents are described by a \emph{directed} network. The D-DPS combines surplus consensus to overcome the asymmetry caused by the directed communication network. The analysis shows the convergence rate to …
We consider a distributed multi-agent network system where the goal is to minimize a sum of convex objective functions of the agents subject to a common convex constraint set. Each agent maintains an iterate sequence and communicates the iterates to its neighbors. Then, each agent combines weighted averages of the received iterates with its own iterate, and adjusts the iterate by using subgradient information (known with stochastic errors) of its own function and by projecting onto the constraint set. The goal of this paper is to explore the effects of stochastic subgradient errors on the conv…
We propose a subgradient-based method for finding the maximum feasible subsystem in a collection of closed sets with respect to a given closed set $C$ (MFS$_C$). In this method, we reformulate the MFS$_C$ problem as an $\ell_0$ optimization problem and construct a sequence of continuous optimization problems to approximate it. The objective of each approximation problem is the sum of the composition of a nonnegative nondecreasing continuously differentiable concave function with the squared distance function to a closed set. Although this objective function is nonsmooth in general, a subgradie…
On The Behavior of Subgradient Projections Methods for Convex Feasibility Problems in Euclidean Spaces
针对不一致情况下一般凸集的凸可行性问题，研究次梯度投影方法，提出一种自适应控制松弛参数的策略，保证算法行为并展示计算优势。
In this paper, we consider a distributed constrained optimization problem with delayed subgradient information over the time-varying communication network, where each agent can only communicate with its neighbors and the communication channel has a limited data rate. We propose an adaptive quantization method to address this problem. A mirror descent algorithm with delayed subgradient information is established based on the theory of Bregman divergence. With non-Euclidean Bregman projection-based scheme, the proposed method essentially generalizes many previous classical Euclidean projection-b…
We consider the problem of minimizing a convex function that depends on an uncertain parameter $θ$. The uncertainty in the objective function means that the optimum, $x^*(θ)$, is also a function of $θ$. We propose an efficient method to compute $x^*(θ)$ and its statistics. We use a chaos expansion of $x^*(θ)$ along a truncated basis and study a restarted subgradient method that compute the optimal coefficients. We establish the convergence rate of the method as the number of basis functions increases, and hence the dimensionality of the optimization problem is increased. We give a non-asymptot…
Stochastic subgradient for composite optimization with functional constraints
本文针对带随机复合目标函数与无限约束交集的凸优化问题，提出带随机可行性更新的随机次梯度方法，证明其在不同情形下的收敛速率，数值结果验证了方法有效性。
Mini-batch stochastic subgradient for functional constrained optimization
针对含多函数约束的有限和复合凸优化问题，提出将其重构为随机优化问题的小批量随机次梯度投影算法，推导步长表达式与切换规则，证明次线性收敛速率且数值性能优于单批量方案。
Projection-Free Algorithms for Nonsmooth Stochastic Convex-Concave Saddle-Point Problems
本文针对非光滑随机凸-凹鞍点问题，提出模块化单循环无投影算法，利用线性最小化预言机直接处理非光滑性，达到O(ε^-2)迭代复杂度并证明极小极大最优性。
Finite Termination of a Generalized Perceptron Algorithm
本文研究了凸不等式系统在希尔伯特空间中的简单次梯度方法，证明在严格可行性与有界次梯度条件下，若干自然步长下算法具有有限终止性，并探讨了无严格可行性时可能存在的收敛失败问题。
We propose first order algorithms for convex optimization problems where the feasible set is described by a large number of convex inequalities that is to be explored by subgradient projections. The first algorithm is an adaptation of a subgradient algorithm, and has convergence rate $1/\sqrt{k}$. The second algorithm has convergence rate 1/k when (1) one has linear metric inequality in the feasible set, (2) the objective function is strongly convex, differentiable and has Lipschitz gradient, and (3) it is easy to optimize the objective function on the intersection of two halfspaces. This seco…
A subgradient method is presented for solving general convex optimization problems, the main requirement being that a strictly-feasible point is known. A feasible sequence of iterates is generated, which converges to within user-specified error of optimality. Feasibility is maintained with a line-search at each iteration, avoiding the need for orthogonal projections onto the feasible region (the operation that limits practicality of traditional subgradient methods). Lipschitz continuity is not required, yet the algorithm is shown to possess a convergence rate analogous to rates for traditional…
Two distributed algorithms are described that enable all users connected over a network to cooperatively solve the problem of minimizing the sum of all users' objective functions over the intersection of all users' constraint sets, where each user has its own private nonsmooth convex objective function and closed convex constraint set, which is the intersection of a number of simple, closed convex sets. One algorithm enables each user to adjust its estimate by using a proximity operator of its objective function and the metric projection onto one set randomly selected from the simple, closed c…
This paper shows that the optimal subgradient algorithm, OSGA, proposed in \cite{NeuO} can be used for solving structured large-scale convex constrained optimization problems. Only first-order information is required, and the optimal complexity bounds for both smooth and nonsmooth problems are attained. More specifically, we consider two classes of problems: (i) a convex objective with a simple closed convex domain, where the orthogonal projection on this feasible domain is efficiently available; (ii) a convex objective with a simple convex functional constraint. If we equip OSGA with an appro…
On subgradient projectors
本文系统研究了子梯度投影器的性质，探讨了其连续性、非扩展性和单调性，并讨论了Yamagishi-Yamada算子，通过实例展示了研究成果。
Nonsmooth Projection-Free Optimization with Functional Constraints
本文提出一种基于次梯度的无投影算法，用于求解带凸函数不等式约束的非光滑凸优化问题，每次迭代仅需一次LMO调用和一次次梯度计算，达到$\mathcal{O}(ε^{-2})$迭代复杂度，并通过分离方案与新拉格朗日乘子更新规则实现。
Constrained quasiconvex optimization problems appear in many fields, such as economics, engineering, and management science. In particular, fractional programming, which models ratio indicators such as the profit/cost ratio as fractional objective functions, is an important instance. Subgradient methods and their variants are useful ways for solving these problems efficiently. Many complicated constraint sets onto which it is hard to compute the metric projections in a realistic amount of time appear in these applications. This implies that the existing methods cannot be applied to quasiconvex…
In this paper we propose a generalized condition for a sharp minimum, somewhat similar to the inexact oracle proposed recently by Devolder-Glineur-Nesterov. The proposed approach makes it possible to extend the class of applicability of subgradient methods with the Polyak step-size, to the situation of inexact information about the value of the minimum, as well as the unknown Lipschitz constant of the objective function. Moreover, the use of local analogs of the global characteristics of the objective function makes it possible to apply the results of this type to wider classes of problems. We…
Projection-Free Non-Smooth Convex Programming
本文提出无投影的非光滑凸优化算法，通过乐观原始-对偶更新和局部线性优化预言机，分别对Lipschitz和强凸目标实现最优误差界，并消除对数因子。
A Hybrid Subgradient Method for Nonsmooth Nonconvex Bilevel Optimization
本文提出一种混合子梯度法，结合双时间尺度动量加速子梯度法和可行性恢复方案，解决非凸双层优化问题，实现全局收敛性和高效计算。
Projected Subgradient Ascent for Convex Maximization
本文提出了一种投影子梯度上升方法用于解决凸最大化问题，证明其在特定条件下收敛于一阶 stationary 点。
We consider the classical setting of optimizing a nonsmooth Lipschitz continuous convex function over a convex constraint set, when having access to a (stochastic) first-order oracle (FO) for the function and a projection oracle (PO) for the constraint set. It is well known that to achieve $ε$-suboptimality in high-dimensions, $Θ(ε^{-2})$ FO calls are necessary. This is achieved by the projected subgradient method (PGD). However, PGD also entails $O(ε^{-2})$ PO calls, which may be computationally costlier than FO calls (e.g. nuclear norm constraints). Improving this PO calls complexity of PGD …
AN-SPS: Adaptive Sample Size Nonmonotone Line Search Spectral Projected Subgradient Method for Convex Constrained Optimization Problems
针对期望形式的非光滑凸约束优化问题，提出自适应样本量的非单调线搜索谱投影次梯度框架AN-SPS，用自适应SAA近似目标函数，证明几乎必然收敛，数值实验验证了方案优势。
We propose a generalization of the method of cyclic projections, which uses the lengths of projection steps carried out in the past to learn about the geometry of the problem and decides on this basis which projections to carry out in the future. We prove the convergence of this algorithm and illustrate its behavior in a first numerical study.
We introduce a relaxed-projection splitting algorithm for solving variational inequalities in Hilbert spaces for the sum of nonsmooth maximal monotone operators, where the feasible set is defined by a nonlinear and nonsmooth continuous convex function inequality. In our scheme, the orthogonal projections onto the feasible set are replaced by projections onto separating hyperplanes. Furthermore, each iteration of the proposed method consists of simple subgradient-like steps, which does not demand the solution of a nontrivial subproblem, using only individual operators, which exploits the struct…
Parallel and cyclic projection algorithms are proposed for minimizing the sum of a finite family of convex functions over the intersection of a finite family of closed convex subsets of a Hilbert space. These algorithms are of predictor-corrector type, with each main iteration consisting of an inner cycle of subgradient descent process followed by a projection step. We prove the convergence of these methods to an optimal solution of the composite minimization problem under investigation upon assuming boundedness of the gradients at the iterates of the local functions and the stepsizes being ch…
Generalized Bregman Projection Algorithms for Solving Nonlinear Split Feasibility Problems in Infinite-Dimensional Spaces
针对无限维希尔伯特空间的非线性分裂可行性问题，提出融合Bregman投影、近端梯度步和自适应惯性项的广义Bregman投影算法，证明强收敛性且实验验证其优于经典方法，推动相关优化技术发展。
Finding a point in the intersection of a collection of closed convex sets, that is the convex feasibility problem, represents the main modeling strategy for many computational problems. In this paper we analyze new stochastic reformulations of the convex feasibility problem in order to facilitate the development of new algorithmic schemes. We also analyze the conditioning problem parameters using certain (linear) regularity assumptions on the individual convex sets. Then, we introduce a general random projection algorithmic framework, which extends to the random settings many existing projecti…
In this work we study the method of Bregman projections for deterministic and stochastic convex feasibility problems with three types of control sequences for the selection of sets during the algorithmic procedure: greedy, random, and adaptive random. We analyze in depth the case of affine feasibility problems showing that the iterates generated by the proposed methods converge Q-linearly and providing also explicit global and local rates of convergence. This work generalizes from one hand recent developments in randomized methods for the solution of linear systems based on orthogonal projecti…
We introduce an abstract algorithm that aims to find the Bregman projection onto a closed convex set. As an application, the asymptotic behaviour of an iterative method for finding a fixed point of a quasi Bregman nonexpansive mapping with the fixed-point closedness property is analyzed. We also show that our result is applicable to Bregman subgradient projectors.
We propose a linesearch projection algorithm for solving non-monotone and non-Lipschitzian equilibrium problems in Hilbert spaces. It is proved that the sequence generated by the proposed algorithm converges strongly to a solution of the equilibrium problem under the assumption that the solution set of the associated Minty equilibrium problem is nonempty. Compared with existing methods, we do not employ Fejér monotonicity in the strategy of proving the convergence. This comes from projecting a fixed point instead of the current point onto a subset of the feasible set at each iteration. Moreove…
Global convergence and acceleration of projection methods for feasibility problems involving union convex sets
本文针对并凸集可行性问题建立经典投影算法的全局收敛理论，并据此推导稀疏仿射可行性与含 $P$-矩阵线性互补问题的收敛条件，同时提出有收敛保证且数值上更高效的加速算法。
We consider the superiorization methodology, which can be thought of as lying between feasibility-seeking and constrained minimization. It is not quite trying to solve the full fledged constrained minimization problem; rather, the task is to find a feasible point which is superior (with respect to the objective function value) to one returned by a feasibility-seeking only algorithm. Our main result reveals new information about the mathematical behavior of the superiorization methodology. We deal with a constrained minimization problem with a feasible region, which is the intersection of finit…
Quadratic Convergence of a Projection Method for a Plane Curve Feasibility Problem
本文证明平面光滑曲线与直线相交可行性问题中投影算法在无切向相交条件下的二次收敛性，并用数值实验验证，为高维推广奠定基础。
A Composed Alternating Relaxed Projection Algorithm for Feasibility Problem
针对可行性问题，提出一种组合交替松弛投影算法，基于道格拉斯-拉赫福德分裂法融入投影与反射组合，设计非平稳版本提升实际性能，提供理论保证并获数值实验支持。
In this paper we present a new algorithmic realization of a projection-based scheme for general convex constrained optimization problem. The general idea is to transform the original optimization problem to a sequence of feasibility problems by iteratively constraining the objective function from above until the feasibility problem is inconsistent. For each of the feasibility problems one may apply any of the existing projection methods for solving it. In particular, the scheme allows the use of subgradient projections and does not require exact projections onto the constraints sets as in exis…
Parallel Polyhedral Projection Method for the Convex Feasibility Problem
本文提出3PM和A3PM方法求解凸可行性问题，通过平行投影与多面体投影两阶段迭代，在无正则性假设下证明全局收敛，并在特定条件下证明线性或超线性收敛，数值实验显示其优于现有方法。
We introduce and analyze an abstract algorithm that aims to find the projection onto a closed convex subset of a Hilbert space. When specialized to the fixed point set of a quasi nonexpansive mapping, the required sufficient condition (termed "fixed-point closed") is less restrictive than the usual conditions based on the demiclosedness principle. A concrete example of a subgradient projector is presented which illustrates the applicability of this generalization.
Constrained Online Convex Optimization with Polyak Feasibility Steps
研究带固定约束的在线凸优化，提出基于Polyak步长的可行性步方法，在仅访问约束值和次梯度下实现任意时刻约束满足且保持 $O(\sqrt{T})$ 遗憾界。
Stochastic Block Bregman Projection with Polyak-like Stepsize for Possibly Inconsistent Convex Feasibility Problems
本文提出了一种统一的随机双层改写方法，结合了接近函数最小化和结构正则化，以解决可能不一致的凸可行性问题，并通过Polyak-like步长实现了期望精确收敛。
The problem of minimizing the sum of nonsmooth, convex objective functions defined on a real Hilbert space over the intersection of fixed point sets of nonexpansive mappings, onto which the projections cannot be efficiently computed, is considered. The use of proximal point algorithms that use the proximity operators of the objective functions and incremental optimization techniques is proposed for solving the problem. With the focus on fixed point approximation techniques, two algorithms are devised for solving the problem. One blends an incremental subgradient method, which is a useful algor…
Let $A$ be a real $M \times N$ measurement matrix and $b\in \mathbb{R}^M$ be an observations vector. The affine feasibility problem with sparsity and nonnegativity ($AFP_{SN}$ for short) is to find a sparse and nonnegative vector $x\in \mathbb{R}^N$ with $Ax=b$ if such $x$ exists. In this paper, we focus on establishment of optimization approach to solving the $AFP_{SN}$. By discussing tangent cone and normal cone of sparse constraint, we give the first necessary optimality conditions, $α$-Stability, T-Stability and N-Stability, and the second necessary and sufficient optimality conditions for…
A modified double inertial subgradient extragradient algorithm for non-monotone variational inequality with applications
本文提出了一种改进的双惯性迭代方法，用于解决非单调变分不等式问题，通过自适应步长规则在不预先知道算子Lipschitz常数的情况下实现弱收敛，并讨论了不同算子类型的收敛性及线性收敛性。
Stochastic Subgradient Descent Escapes Active Strict Saddles on Weakly Convex Functions
该研究在非光滑随机优化中证明，在可定义弱凸函数类中通有成立的Verdier分层与角度条件下，随机次梯度下降不会收敛到主动严格鞍点，而会收敛到局部极小值点。
The circumcentered-reflection method (CRM) has been applied for solving convex feasibility problems. CRM iterates by computing a circumcenter upon a composition of reflections with respect to convex sets. Since reflections are based on exact projections, their computation might be costly. In this regard, we introduce the circumcentered approximate-reflection method (CARM), whose reflections rely on outer-approximate projections. The appeal of CARM is that, in rather general situations, the approximate projections we employ are available under low computational cost. We derive convergence of CA…
We point out that Chubanov's oracle-based algorithm for linear programming [5] can be applied almost as it is to linear semi-infinite programming (LSIP). In this note, we describe the details and prove the polynomial complexity of the algorithm based on the real computation model proposed by Blum, Shub and Smale (the BSS model) which is more suitable for floating point computation in modern computers. The adoption of the BBS model makes our description and analysis much simpler than the original one by Chubanov [5]. Then we reformulate semidefinite programming (SDP) and second-order cone progr…
We analyze a simple randomized subgradient method for approximating solutions to stochastic systems of convex functional constraints, the only input to the algorithm being the size of minibatches. By introducing a new notion of what is meant for a point to approximately solve the constraints, determining bounds on the expected number of iterations reduces to determining a hitting time for a compound Bernoulli process, elementary probability. Besides bounding the expected number of iterations quite generally, we easily establish concentration inequalities on the number of iterations, and more i…
A Busemann hybrid projection-proximal point algorithm for optimization problems on Hadamard manifolds
本文提出了一种基于Busemann函数的混合投影-近点算法，用于Hadamard流形上的优化问题，实现了几何内禀的迭代方案并证明了全局收敛性。
扫码添加微信好友，提出您的宝贵建议 👇
💡 备注请填写：网站反馈
发表机构由 AI 从论文文本抽取，如发现错误，欢迎反馈修正建议。