The Karush-Kuhn-Tucker and value function (lower-level value function, to be precise) reformulations are the most common single-level transformations of the bilevel optimization problem. So far, these reformulations have either been studied independently or as a joint optimization problem in an attempt to take advantage of the best properties from each model. To the best of our knowledge, these reformulations have not yet been compared in the existing literature. This paper is a first attempt towards establishing whether one of these reformulations is best at solving a given class of the optimistic bilevel optimization problem. We design a comparison framework, which seems fair, considering the theoretical properties of these reformulations. This work reveals that although none of the models seems to particularly dominate the other from the theoretical point of view, the value function reformulation seems to numerically outperform the Karush-Kuhn-Tucker reformulation on a Newton-type algorithm. The computational experiments here are mostly based on test problems from the Bilevel Optimization LIBrary (BOLIB).
We provide a simple and short proof of the Karush-Kuhn-Tucker theorem with finite number of equality and inequality constraints. The proof relies on an elementary linear algebra lemma and the local inverse theorem.
In the recent paper of Giorgi, Jiménez and Novo (J Optim Theory Appl 171:70--89, 2016), the authors introduced the so-called approximate Karush-Kuhn-Tucker (AKKT) condition for smooth multiobjective optimization problems and obtained some AKKT-type necessary optimality conditions and sufficient optimality conditions for weak efficient solutions of such a problem. In this note, we extend these optimality conditions to locally Lipschitz multiobjective optimization problems using Mordukhovich subdifferentials. Furthermore, we prove that, under some suitable additional conditions, an AKKT conditio…
In the present paper, we focus on the vector optimization problems with inequality constraints, where objective functions and constrained functions are Fréchet differentiable, and whose gradient mapping is locally Lipschitz on an open set. By using the second-order symmetric subdifferential and the second-order tangent set, we propose two types of second-order regularity conditions in the sense of Abadie. Then we establish some strong second-order Karush--Kuhn--Tucker necessary optimality conditions for Geoffrion properly efficient solutions of the considered problem. Examples are given to ill…
Kuhn-Tucker conditions for mathematical programming problems in Banach spaces partially ordered by cone with empty interior are obtained under strong simultaneity condition. If partial ordered cone has interior point, it is proved that Slater and strong simultaneity conditions are equivalent.
Kuhn-Tucker points play a fundamental role in the analysis and the numerical solution of monotone inclusion problems, providing in particular both primal and dual solutions. We propose a class of strongly convergent algorithms for constructing the best approximation to a reference point from the set of Kuhn-Tucker points of a general Hilbertian composite monotone inclusion problem. Applications to systems of coupled monotone inclusions are presented. Our framework does not impose additional assumptions on the operators present in the formulation, and it does not require knowledge of the norm o…
Some necessary and sufficient optimality conditions for inequality constrained problems with continuously differentiable data were obtained in the papers [I. Ginchev and V.I. Ivanov, Second-order optimality conditions for problems with C$\sp{1}$ data, J. Math. Anal. Appl., v. 340, 2008, pp. 646--657], [V.I. Ivanov, Optimality conditions for an isolated minimum of order two in C$\sp{1}$ constrained optimization, J. Math. Anal. Appl., v. 356, 2009, pp. 30--41] and [V. I. Ivanov, Second- and first-order optimality conditions in vector optimization, Internat. J. Inform. Technol. Decis. Making, 201…
We introduce the first operator splitting method for composite monotone inclusions outside of Hilbert spaces. The proposed primal-dual method constructs iteratively the best Bregman approximation to an arbitrary point from the Kuhn-Tucker set of a composite monotone inclusion. Strong convergence is established in reflexive Banach spaces without requiring additional restrictions on the monotone operators or knowledge of the norms of the linear operators involved in the model. The monotone operators are activated via Bregman distance-based resolvent operators. The method is novel even in Euclide…
We propose a new class of primal-dual Fejer monotone algorithms for solving systems of com- posite monotone inclusions. Our construction is inspired by a framework used by Eckstein and Svaiter for the basic problem of finding a zero of the sum of two monotone operators. At each iteration, points in the graph of the monotone operators present in the model are used to con- struct a half-space containing the Kuhn-Tucker set associated with the system. The primal-dual update is then obtained via a relaxed projection of the current iterate onto this half-space. An important feature that distinguish…
This expository paper contains a concise introduction to some significant works concerning the Karush-Kuhn-Tucker condition, a necessary condition for a solution in local optimality in problems with equality and inequality constraints. The study of this optimality condition has a long history and culminated in the appearance of subdifferentials. The 1970s and early 1980s were important periods for new developments and various generalizations of subdifferentials were introduced, including the Clarke subdifferential and Demyanov-Rubinov quasidifferential. In this paper, we mainly present four ge…
We extend in two ways the standard Karush-Kuhn-Tucker optimality conditions to problems with a convex objective, convex functional constraints, and the extra requirement that some of the variables must be integral. While the standard Karush-Kuhn-Tucker conditions involve separating hyperplanes, our extension is based on lattice-free polyhedra. Our optimality conditions allow us to define an exact dual of our original mixed-integer convex problem.
In this paper we study a continuous time, optimal stochastic investment problem under limited resources in a market with N firms. The investment processes are subject to a time-dependent stochastic constraint. Rather than using a dynamic programming approach, we exploit the concavity of the profit functional to derive some necessary and sufficient first order conditions for the corresponding Social Planner optimal policy. Our conditions are a stochastic infinite-dimensional generalization of the Kuhn-Tucker Theorem. The Lagrange multiplier takes the form of a nonnegative optional random measur…
In this paper we obtain second- and first-order optimality conditions of Kuhn-Tucker type and Fritz John one for weak efficiency in the vector problem with inequality constraints. In the necessary conditions we suppose that the objective function and the active constraints are continuously differentiable. We introduce notions of KTSP-invex problem and second-order KTSP-invex one. We obtain that the vector problem is (second-order) KTSP-invex if and only if for every triple $(\bar x,\barλ,\barμ)$ with Lagrange multipliers $\barλ$ and $\barμ$ for the objective function and constraints respective…
Although the Karush-Kuhn-Tucker conditions suggest a connection between a conic optimization problem and a complementarity problem, it is difficult to find an accessible explicit form of this relationship in the literature. This note will present such a relationship.
This paper deals with approximate solutions of an optimization problem with interval-valued objective function. Four types of approximate solution concepts of the problem are proposed by considering the partial ordering $LU$ on the set of all closed and bounded intervals. We show that these solutions exist under very weak conditions. Under suitable constraint qualifications, we derive Karush--Kuhn--Tucker necessary and sufficient optimality conditions for convex interval-valued optimization problems.
This paper pursues a two-fold goal. Firstly, we aim to derive novel second-order characterizations of important robust stability properties of perturbed Karush-Kuhn-Tucker systems for a broadclass of constrained optimization problems generated by parabolically regular sets. Secondly, the obtained characterizations are applied to establish well-posedness and superlinear convergence of the basic sequential quadratic programming method to solve parabolically regular constrained optimization problems.
In this paper, we introduce a kind of approximate Karush--Kuhn--Tucker condition (AKKT) for a smooth cone-constrained vector optimization problem. We show that, without any constraint qualification, the AKKT condition is a necessary for a local weak efficient solution of the considered problem. For convex problems, we prove that the AKKT condition is a necessary and sufficient optimality condition for a global weak efficient solution. We also introduce some strict constraint qualifications associated with the AKKT condition.
By applying the perturbation function approach, we propose the Lagrangian and the conjugate duals for minimization problems of the sum of two, generally nonconvex, functions. The main tools are the $Φ$-convexity theory and minimax theorems for $Φ$-convex functions. We provide conditions ensuring zero duality gap and introduce $Φ$-Karush-Kuhn-Tucker conditions that characterize solutions to primal and dual problems. We also discuss the relationship between the dual problems introduced in the present investigation and some conjugate-type duals existing in the literature.
A Sequential Quadratic Programming Method for Optimization with Stochastic Objective Functions, Deterministic Inequality Constraints and Robust Subproblems
The paper demonstrates the equivalence between the optimality criteria (OC) method, initially proposed by Bendsoe & Kikuchi for topology optimization problem, and the projected gradient method. The equivalence is shown using Hestenes definition of Lagrange multipliers. Based on this development, an alternative formulation of the Karush-Kuhn-Tucker (KKT) condition is suggested. Such reformulation has some advantages, which will be also discussed in the paper. For verification purposes the modified algorithm is applied to the design of tension-only structures.
We consider a bilevel program involving a linear lower level problem with left-hand-side perturbation. We then consider the Karush-Kuhn-Tucker reformulation of the problem and subsequently build a tractable optimization problem with linear constraints by means of a partial exact penalization. A semismooth system of equations is then generated from the later problem and a Newton-type method is developed to solve it. Finally, we illustrate the convergence and practical implementation of the algorithm on the optimal toll-setting problem in transportation networks.
The purpose of this paper is to characterize the weak efficient solutions, the efficient solutions, and the isolated efficient solutions of a given vector optimization problem with finitely many convex objective functions and infinitely many convex constraints. To do this, we introduce new and already known data qualifications (conditions involving the constraints and/or the objectives) in order to get optimality conditions which are expressed in terms of either Karusk-Kuhn-Tucker multipliers or a new gap function associated with the given problem.
Given a non-convex optimization problem, we study conditions under which every Karush-Kuhn-Tucker (KKT) point is a global optimizer. This property is known as KT-invexity and allows to identify the subset of problems where an interior point method always converges to a global optimizer. In this work, we provide necessary conditions for KT-invexity in n-dimensions and show that these conditions become sufficient in the two-dimensional case. As an application of our results, we study the Optimal Power Flow problem, showing that under mild assumptions on the variable's bounds, our new necessary a…
Partial calmness is a celebrated but restrictive property of bilevel optimization problems whose presence opens a way to the derivation of Karush--Kuhn--Tucker-type necessary optimality conditions in order to characterize local minimizers. In the past, sufficient conditions for the validity of partial calmness have been investigated. In this regard, the presence of a linearly structured lower level problem has turned out to be beneficial. However, the associated literature suffers from inaccurate results. In this note, we clarify some regarding erroneous statements and visualize the underlying…
In the present paper, we are concerned with a class of constrained vector optimization problems, where the objective functions and active constraint functions are locally Lipschitz at the referee point. Some second-order constraint qualifications of Zangwill type, Abadie type and Mangasarian~--~Fromovitz type as well as a regularity condition of Abadie type are proposed in a nonsmooth setting. The connections between these proposed conditions are established. They are applied to develop second-order Karush--Kuhn--Tucker necessary optimality conditions for local (weak, Geoffrion properly) effic…
This paper studies bilevel polynomial optimization problems. To solve them, we give a method based on polynomial optimization relaxations. Each relaxation is obtained from the Kurash-Kuhn-Tucker (KKT) conditions for the lower level optimization and the exchange technique for semi-infinite programming. For KKT conditions, Lagrange multipliers are represented as polynomial or rational functions. The Moment-SOS relaxations are used to solve the polynomial optimizattion relaxations. Under some general assumptions, we prove the convergence of the algorithm for solving bilevel polynomial optimizatio…
We establish a geometric condition guaranteeing exact copositive relaxation for the nonconvex quadratic optimization problem under two quadratic and several linear constraints, and present sufficient conditions for global optimality in terms of generalized Karush-Kuhn-Tucker multipliers. The copositive relaxation is tighter than the usual Lagrangian relaxation. We illustrate this by providing a whole class of quadratic optimization problems that enjoys exactness of copositive relaxation while the usual Lagrangian duality gap is infinite. Finally, we also provide verifiable conditions under whi…
Constrained Optimization From a Control Perspective via Feedback Linearization
Massachusetts Institute of Technology（麻省理工学院）；Mitsubishi Electric Research Laboratories（三菱电机研究实验室）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Harvard University（哈佛大学）
In this article, we introduce a new homotopy function to trace the trajectory by applying modified homotopy continuation method for finding the solution of the linear complementarity problem. Earlier several authors attempted to propose homotopy functions based on original problems. We propose the homotopy function based on the Karush-Kuhn-Tucker condition of the corresponding quadratic programming problem. The proposed approach extends the processability of the larger class of linear complementarity problem and overcomes the limitations of other existing homotopy approaches. We show that the …
We extend the convergence analysis of the Scholtes-type regularization method for cardinality-constrained optimization problems. Its behavior is clarified in the vicinity of saddle points, and not just of minimizers as it has been done in the literature before. This becomes possible by using as an intermediate step the recently introduced regularized continuous reformulation of a cardinality-constrained optimization problem. We show that the Scholtes-type regularization method is well-defined locally around a nondegenerate T-stationary point of this regularized continuous reformulation. Moreov…
In this paper, we study variational inequality problems (VIPs) with involved mappings and feasible sets characterized by polynomial functions (namely, polynomial VIPs). We propose a numerical algorithm for computing solutions to polynomial VIPs based on Lagrange multiplier expressions and the Moment-SOS hierarchy of semidefinite relaxations. We also extend our approach to finding more or even all solutions to polynomial VIPs. We show that the method proposed in this paper can find solutions or detect the nonexistence of solutions within finitely many steps, under some general assumptions. In a…
Recently, Yamanaka and Yamashita proposed the so-called positively homogeneous optimization problem, which includes many important problems, such as the absolute-value and the gauge optimizations. They presented a closed form of the dual formulation for the problem, and showed weak duality and the equivalence to the Lagrangian dual under some conditions. In this work, we focus on a special positively homogeneous optimization problem, whose objective function and constraints consist of some gauge and linear functions. We prove not only weak duality but also strong duality. We also study necessa…
We consider the convex optimization problem P: min {f(x): x in K} where "f" is convex continuously differentiable, and K is a compact convex set in Rn with representation {x: g_j(x) >=0, j=1,;;,m} for some continuously differentiable functions (g_j). We discuss the case where the g_j's are not all concave (in contrast with convex programming where they all are). In particular, even if the g_j's are not concave, we consider the log-barrier function phi_μwith parameter μ, associated with P, usually defined for concave functions (g_j). We then show that any limit point of any sequence (x_μ) of st…
Minimax optimization problems arises from both modern machine learning including generative adversarial networks, adversarial training and multi-agent reinforcement learning, as well as from tradition research areas such as saddle point problems, numerical partial differential equations and optimality conditions of equality constrained optimization. For the unconstrained continuous nonconvex-nonconcave situation, Jin, Netrapalli and Jordan (2019) carefully considered the very basic question: what is a proper definition of local optima of a minimax optimization problem, and proposed a proper de…
Sample-Based Consistency in Infinite-Dimensional Conic-Constrained Stochastic Optimization
University of Klagenfurt（克拉根福大学）；H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology（佐治亚理工学院 H. 米尔顿·斯图尔特工业与系统工程学校）
An effective means for analyzing the impact of novel operating schemes on power systems is time domain simulation, for example for investigating optimization-based curtailment of renewables to alleviate voltage violations. Traditionally, interior-point methods are used for solving the non-convex AC optimal power flow (OPF) problems arising in this type of simulation. This paper presents an alternative algorithm that better suits the simulation framework, because it can more effectively be warm-started, has linear computational and memory complexity in the problem size per iteration and globall…
The Environmental and Economic Dispatch problem (EEDP)is a nonlinear Multi-objective Optimization Problem (MOP) which simultaneously satisfies multiple contradictory criteria, and it's a nonsmooth problem when valvepoint effects, multi-fuel effects and prohibited operating zones have been considered. It is an important optimization task in fossil fuel fired power plant operation for allocating generation among the committed units such that fuel cost and pollution (emission level) are optimized simultaneously while satisfying all operational constraints. In this paper, we use smoothing function…
In this paper, we provide a complete characterization on the robust isolated calmness of the Karush-Kuhn-Tucker (KKT) solution mapping for convex constrained optimization problems regularized by the nuclear norm function. This study is motivated by the recent work in [8], where the authors show that under the Robinson constraint qualification at a local optimal solution, the KKT solution mapping for a wide class of conic programming problem is robustly isolated calm if and only if both the second order sufficient condition (SOSC) and the strict Robinson constraint qualification (SRCQ) are sati…
We consider optimization problems on Riemannian manifolds with equality and inequality constraints, which we call Riemannian nonlinear optimization (RNLO) problems. Although they have numerous applications, the existing studies on them are limited especially in terms of algorithms. In this paper, we propose Riemannian sequential quadratic optimization (RSQO) that uses a line-search technique with an ell_1 penalty function as an extension of the standard SQO algorithm for constrained nonlinear optimization problems in Euclidean spaces to Riemannian manifolds. We prove its global convergence to …
This paper considers a nonconvex optimization problem that evolves over time, and addresses the synthesis and analysis of regularized primal-dual gradient methods to track a Karush-Kuhn-Tucker (KKT) trajectory. The proposed regularized primal-dual gradient methods are implemented in a running fashion, in the sense that the underlying optimization problem changes during the iterations of the algorithms. For a problem with twice continuously differentiable cost and constraints, and under a generalization of the Mangasarian-Fromovitz constraint qualification, sufficient conditions are derived for…
This paper addresses the issue of data injection attacks on control systems. We consider attacks which aim at maximizing system disruption while staying undetected in the finite horizon. The maximum possible disruption caused by such attacks is formulated as a non-convex optimization problem whose dual problem is a convex semi-definite program. We show that the duality gap is zero using S-lemma. To determine the optimal attack vector, we formulate a soft-constrained optimization problem using the Lagrangian dual function. The framework of dynamic programming for indefinite cost functions is us…
In this note we present a multigrid preconditioning method for solving quadratic optimization problems constrained by a fractional diffusion equation. Multigrid methods within the all-at-once approach to solve the first order-order optimality Karush-Kuhn-Tucker (KKT) systems are widely popular, but their development have relied on the underlying systems being sparse. On the other hand, for most discretizations, the matrix representation of fractional operators is expected to be dense. We develop a preconditioning strategy for our problem based on a reduced approach, namely we eliminate the sta…
