Existence of augmented Lagrange multipliers: reduction to exact penalty functions and localization principle
M. V. Dolgopolik
备注：
This is a slightly edited verion of a pre-print of an article published in Mathematical Programming. The final authenticated version is available online at: https://doi.org/10.1007/s10107-017-1122-y In this version, a mistake in the proof of Theorem 4 was corrected
In this article, we present new general results on existence of augmented Lagrange multipliers. We define a penalty function associated with an augmented Lagrangian, and prove that, under a certain growth assumption on the augmenting function, an augmented Lagrange multiplier exists if and only if this penalty function is exact. We also develop a new general approach to the study of augmented Lagrange multipliers called the localization principle. The localization principle allows one to study the local behaviour of the augmented Lagrangian near globally optimal solutions of the initial optimization problem in order to prove the existence of augmented Lagrange multipliers.
Based on a characterization of the optimality of a feasible solution of a convex entropy minimization problem, one shows that the feasible solutions obtained using formally the Lagrange multipliers method are optimal.
The concept of uniform tangent sets was introduced and discussed in [3 - Krastanov, Ribarska, SIAM J. Control Optim., 55(3), 2017]. This study is devoted to their further investigation and to generalization of the abstract Lagrange multiplier rule from [3].
In this note, three Lagrange multiplier rules introduced in the literature for set valued optimization problems are compared. A generalization of all three results is given which proves that under rather mild assumptions, $x$ is a weak solution to the constrained problem, if and only if it is a weak solution to the Lagrangian.
In this work we provide a first order sensitivity analysis of some parameterized stochastic optimal control problems. The parameters can be given by random processes. The main tool is the one-to-one correspondence between the adjoint states appearing in a weak form of the stochastic Pontryagin principle and the Lagrange multipliers associated to the state equation.
We use an extension to the infinite dimension of the rank theorem of the differential calculus to establish a Karush-Huhn-Tucker theorem for optimization problems in Banach spaces. We provide an application to variational problems on bounded processus under equality constraints.
We analyze sequences generated by interior point methods (IPMs) in convex and nonconvex settings. We prove that moving the primal feasibility at the same rate as the barrier parameter $μ$ ensures the Lagrange multiplier sequence remains bounded, provided the limit point of the primal sequence has a Lagrange multiplier. This result does not require constraint qualifications. We also guarantee the IPM finds a solution satisfying strict complementarity if one exists. On the other hand, if the primal feasibility is reduced too slowly, then the algorithm converges to a point of minimal complementar…
This paper proposes tight semidefinite relaxations for polynomial optimization. The optimality conditions are investigated. We show that generally Lagrange multipliers can be expressed as polynomial functions in decision variables over the set of critical points. The polynomial expressions can be determined by linear equations. Based on these expressions, new Lasserre type semidefinite relaxations are constructed for solving polynomial optimization. We show that the hierarchy of new relaxations has finite convergence, or equivalently, the new relaxations are tight for a finite relaxation order…
We discuss first order optimality conditions for geometric optimization problems with Neumann boundary conditions and boundary observation. The methods we develop here are applicable to large classes of state systems or cost functionals. Our approach is based on the implicit parametrization theorem and the use of Hamiltonian systems. It establishes equivalence with a constrained optimal control problem and uses Lagrange multipliers under a new simple constraint qualification. In this setting, general functional variations are performed, that combine topological and boundary variations in a nat…
In this paper we apply an augmented Lagrange method to a class of semilinear elliptic optimal control problems with pointwise state constraints. We show strong convergence of subsequences of the primal variables to a local solution of the original problem as well as weak convergence of the adjoint states and weak* convergence of the multipliers associated to the state constraint. Moreover, we show existence of stationary points in arbitrary small neighborhoods of local solutions of the original problem. Additionally, various numerical results are presented.
The paper concerns the study of criticality of Lagrange multipliers in variational systems that has been recognized in both theoretical and numerical aspects of optimization and variational analysis. In contrast to the previous developments dealing with polyhedral KKT systems and the like, we now focus on general nonpolyhedral systems that are associated, in particular, with problems of conic programming. Developing a novel approach, which is mainly based on advanced techniques and tools of second-order variational analysis and generalized differentiation, allows us to overcome principal chall…
This paper studies bilevel polynomial optimization problems. To solve them, we give a method based on polynomial optimization relaxations. Each relaxation is obtained from the Kurash-Kuhn-Tucker (KKT) conditions for the lower level optimization and the exchange technique for semi-infinite programming. For KKT conditions, Lagrange multipliers are represented as polynomial or rational functions. The Moment-SOS relaxations are used to solve the polynomial optimizattion relaxations. Under some general assumptions, we prove the convergence of the algorithm for solving bilevel polynomial optimizatio…
The paper is devoted to the study and applications of criticality of Lagrange multipliers in variational systems, which are associated with the class of problems in composite optimization known as extended nonlinear programming (ENLP). The importance of both ENLP and the concept of multiplier criticality in variational systems has been recognized in theoretical and numerical aspects of optimization and variational analysis, while the criticality notion has never been investigated in the ENLP framework. We present here a systematic study of critical and noncritical multipliers in a general vari…
In this paper, we consider the problem of reducing network delay in stochastic network utility optimization problems. We start by studying the recently proposed quadratic Lyapunov function based algorithms (QLA). We show that for every stochastic problem, there is a corresponding \emph{deterministic} problem, whose dual optimal solution "exponentially attracts" the network backlog process under QLA. In particular, the probability that the backlog vector under QLA deviates from the attractor is exponentially decreasing in their Euclidean distance. This not only helps to explain how QLA achieves…
In this paper, we study variational inequality problems (VIPs) with involved mappings and feasible sets characterized by polynomial functions (namely, polynomial VIPs). We propose a numerical algorithm for computing solutions to polynomial VIPs based on Lagrange multiplier expressions and the Moment-SOS hierarchy of semidefinite relaxations. We also extend our approach to finding more or even all solutions to polynomial VIPs. We show that the method proposed in this paper can find solutions or detect the nonexistence of solutions within finitely many steps, under some general assumptions. In a…
Solutions to nonlinear, nonconvex optimization problems can fail to satisfy the KKT optimality conditions even when they are optimal. This is due to the fact that unless constraint qualifications (CQ) are satisfied, Lagrange multipliers may fail to exist. Even if the KKT conditions are applicable, the multipliers may not be unique. These possibilities also affect AC optimal power flow (OPF) problems which are routinely solved in power systems planning, scheduling and operations. The complex structure -- in particular the presence of the nonlinear power flow equations which naturally exhibit a …
We examine two central regularization strategies for monotone variational inequalities, the first a direct regularization of the operative monotone mapping, and the second via regularization of the associated dual gap function. A key link in the relationship between the solution sets to these various regularized problems is the idea of exact regularization, which, in turn, is fundamentally associated with the existence of Lagrange multipliers for the regularized variational inequality. A regularization is said to be exact if a solution to the regularized problem is a solution to the unregulari…
We show how a stochastic version of the Lagrange multiplier method can be combined with the stochastic maximum principle for jump diffusions to solve certain constrained stochastic optimal control problems. Two different terminal constraints are considered; one constraint holds in expectation and the other almost surely. As an application of this method, we study the effects of inflation- and wage risk on optimal consumption. To do this, we consider the optimal consumption problem for a budget constrained agent with a Lévy income process and stochastic inflation. The agent must choose a consum…
Second-order optimality conditions for vector nonlinear programming problems with inequality constraints are studied in this paper. We introduce a new second-order constraint qualification, which includes Mangasarian-Fromovitz constraint qualification as a particular case. We obtain necessary and sufficient conditions for weak efficiency of problems with a second-order pseudoconvex vector objective function and quasiconvex constraints. We also derive Lagrange multiplier characterizations of the solution set of a scalar problem with a second-order pseudoconvex objective function and quasiconvex…
Variational problems under uniform quasiconvex constraints on the gradient are studied. In particular, existence of solutions to such problems is proved as well as existence of lagrange multipliers associated to the uniform constraint. They are shown to satisfy an Euler-Lagrange equation and a complementarity property. Our technique consists in approximating the original problem by a one-parameter family of smooth unconstrained optimization problems. Numerical experiments confirm the ability of our method to accurately compute solutions and Lagrange multipliers.
On error bounds and noncritical Lagrange multipliers
Johann Radon Institute for Computational and Applied Mathematics（Johann Radon计算与应用数学研究所）；Dresden University of Technology, Institute of Numerical Mathematics（德累斯顿工业大学数值数学研究所）
There has been a recent surge of interest in physics-based solvers for combinatorial optimization problems. We present a dynamical solver for the Ising problem that is comprised of a network of coupled parametric oscillators and show that it implements Lagrange multiplier constrained optimization. We show that the pump depletion effect, which is intrinsic to parametric oscillators, enforces binary constraints and enables the system's continuous analog variables to converge to the optimal binary solutions to the optimization problem. Moreover, there is an exact correspondence between the equati…
In this paper, we establish a general result on spherical maxima sharing the same Lagrange multiplier of which the following is a particular consequence: Let $X$ be a real Hilbert space. For each $r>0$, let $S_r=\{x\in X : \|x\|^2=r\}$. Let $J:X\to {\bf R}$ be a sequentially weakly upper semicontinuous functional which is Gâteaux differentiable in $X\setminus \{0\}$. Assume that $$\limsup_{x\to 0}{{J(x)}\over {\|x\|^2}}=+\infty\ .$$ Then, for each $ρ>0$, there exists an open interval $I\subseteq ]0,+\infty[$ and an increasing function $φ:I\to ]0,ρ[$ such that, for each $λ\in I$, one has $$\emp…
In this paper we consider three minimization problems, namely quadratic, $ρ$-convex and quadratic fractional programing problems. The quadratic problem is considered with quadratic inequality constraints with bounded continuous and discrete mixed variables. The $ρ$-convex problem is considered with $ρ$-convex inequality constraints in mixed variables. The quadratic fractional problem is studied with quadratic fractional constraints in mixed variables. For all three problems we reformulate the problem as a mathematical programming problem and apply standard Karush Kuhn Tucker necessary conditio…
We provide a modified augmented Lagrange method coupled with a Tikhonov regularization for solving ill-posed state-constrained elliptic optimal control problems with sparse controls. We consider a linear quadratic optimal control problem without any additional $L^2$ regularization terms. The sparsity is guaranteed by an additional $L^1$ term. Here, the modification of the classical augmented Lagrange method guarantees us uniform boundedness of the multiplier that corresponds to the state constraints. We present a coupling between the regularization parameter introduced by the Tikhonov regulari…
In this paper we present a complete iteration complexity analysis of inexact first order Lagrangian and penalty methods for solving cone constrained convex problems that have or may not have optimal Lagrange multipliers that close the duality gap. We first assume the existence of optimal Lagrange multipliers and study primal-dual first order methods based on inexact information and augmented Lagrangian smoothing or Nesterov type smoothing. For inexact (fast) gradient augmented Lagrangian methods we derive a total computational complexity of $\mathcal{O}\left(\frac{1}ε\right)$ projections onto …
Constrained optimization problems exist in many domains of science, such as thermodynamics, mechanics, economics, etc. These problems are classically solved with the help of the Lagrange multipliers and the Lagrangian function. However, the disadvantage of this approach is that it artificially increases the dimensionality of the problem. Here, we show that the determinant of the Jacobian of the problem (function to optimize and constraints) is null. This extra equation transforms any equality-constrained optimization problem into a solving problem of same dimension. We also introduced the cons…
This paper deals with the geometric numerical integration of gradient flow and its application to optimization. Gradient flows often appear as model equations of various physical phenomena, and their dissipation laws are essential. Therefore, dissipative numerical methods, which are numerical methods replicating the dissipation law, have been studied in the literature. Recently, Cheng, Liu, and Shen proposed a novel dissipative method, the Lagrange multiplier approach, for gradient flows, which is computationally cheaper than existing dissipative methods. Although their efficacy is numerically…
We propose a framework to use Nesterov's accelerated method for constrained convex optimization problems. Our approach consists of first reformulating the original problem as an unconstrained optimization problem using a continuously differentiable exact penalty function. This reformulation is based on replacing the Lagrange multipliers in the augmented Lagrangian of the original problem by Lagrange multiplier functions. The expressions of these Lagrange multiplier functions, which depend upon the gradients of the objective function and the constraints, can make the unconstrained penalty funct…
