The paper is devoted to obtain first and second order necessary optimality conditions for continuous-time optimization problems with equality and inequality constraints. A full rank type regularity condition along with an uniform implicit function theorem are used in order to establish such necessary conditions.
We provide a simple and short proof of the Karush-Kuhn-Tucker theorem with finite number of equality and inequality constraints. The proof relies on an elementary linear algebra lemma and the local inverse theorem.
The aim of this note is to give a geometric insight into the classical second order optimality conditions for equality-constrained minimization problem. We show that the Hessian's positivity of the Lagrangian function associated to the problem at a local minimum point $x^*$ corresponds to inequalities between the respective algebraic curvatures at point $x^*$ of the hypersurface $\mathcal{M}_{f, x^*}=\{ x \in \R^n \, | \, f(x) = f(x^*)\}$ defined by the objective function $f$ and the submanifold $\mathcal{M}_g = \{ x \in \R^n \, | \, g(x)= 0 \}$ defining the contraints. These inequalities high…
In this article we propose a descent method for equality and inequality constrained multiobjective optimization problems (MOPs) which generalizes the steepest descent method for unconstrained MOPs by Fliege and Svaiter to constrained problems by using two active set strategies. Under some regularity assumptions on the problem, we show that accumulation points of our descent method satisfy a necessary condition for local Pareto optimality. Finally, we show the typical behavior of our method in a numerical example.
A sequential quadratic optimization algorithm for minimizing an objective function defined by an expectation subject to nonlinear inequality and equality constraints is proposed, analyzed, and tested. The context of interest is when it is tractable to evaluate constraint function and derivative values in each iteration, but it is intractable to evaluate the objective function or its derivatives in any iteration, and instead an algorithm can only make use of stochastic objective gradient estimates. Under loose assumptions, including that the gradient estimates are unbiased, the algorithm is pro…
The alternating direction method of multipliers (ADMM) proposed by Glowinski and Marrocco is a benchmark algorithm for two-block separable convex optimization problems with linear equality constraints. It has been modified, specified, and generalized from various perspectives to tackle more concrete or complicated application problems. Despite its versatility and phenomenal popularity, it remains unknown whether or not the ADMM can be extended to separable convex optimization problems with linear inequality constraints. In this paper, we lay down the foundation of how to extend the ADMM to two…
Both constrained and unconstrained optimization problems regularly appear in recursive tracking problems engineers currently address -- however, constraints are rarely exploited for these applications. We define the Kalman Filter and discuss two different approaches to incorporating constraints. Each of these approaches are first applied to equality constraints and then extended to inequality constraints. We discuss methods for dealing with nonlinear constraints and for constraining the state prediction. Finally, some experiments are provided to indicate the usefulness of such methods.
Characterization results for equality cases and for rigidity of equality cases in Steiner's perimeter inequality are presented. (By rigidity, we mean the situation when all equality cases are vertical translations of the Steiner's symmetral under consideration.) We achieve this through the introduction of a suitable measure-theoretic notion of connectedness and a fine analysis of barycenter functions for sets of finite perimeter having segments as orthogonal sections with respect to an hyperplane.
This article is devoted to the analysis of necessary and/or sufficient conditions for metric regularity in terms of Demyanov-Rubinov-Polyakova quasidifferentials. We obtain new necessary and sufficient conditions for the local metric regularity of a multifunction in terms of quasidifferentials of the distance function to this multifunction. We also propose a new MFCQ-type constraint qualification for a parametric system of quasidifferentiable equality and inequality constraints and prove that it ensures the metric regularity of a multifunction associated with this system. As an application, we…
We analyze a potentially risk-averse convex stochastic optimization problem, where the control is deterministic and the state is a Banach-valued essentially bounded random variable. We obtain strong forms of necessary and sufficient optimality conditions for problems subject to equality and conical constraints. We propose a Moreau--Yosida regularization for the conical constraint and show consistency of the optimality conditions for the regularized problem as the regularization parameter is taken to infinity.
This paper discusses a special kind of convex constrained optimization problem, whose constraints consist of box inequalities and linear equalities. For this problem, in addition to general optimization algorithms such as exact penalty algorithm and interior point algorithm, there is a simple iterative algorithm that is simple to implement, which is favored by machine learning practitioners.
Optimality conditions in the form of a variational inequality are proved for a class of constrained optimal control problems of stochastic differential equations. The cost function and the inequality constraints are functions of the probability distribution of the state variable at the final time. The analysis uses in an essential manner a convexity property of the set of reachable probability distributions. An augmented Lagrangian method based on the obtained optimality conditions is proposed and analyzed for solving iteratively the problem. At each iteration of the method, a standard stochas…
In this paper, generalizing to the non smooth case already existing results, we prove that, for any convex planar set $Ω$, the first non-trivial Neumann eigenvalue $μ_1(Ω)$ of the Hermite operator is greater than or equal to 1. Furthermore, and this is our main result, under some additional assumptions on $Ω$, we show that $μ_1(Ω)=1$ if and only if $Ω$ is any strip. The study of the equality case requires, among other things, an asymptotic analysis of the eigenvalues of the Hermite operator in thin domains.
In this paper, a kind of neural network with time-varying delays is proposed to solve the problems of quadratic programming. The delay term of the neural network changes with time t. The number of neurons in the neural network is n + h, so the structure is more concise. The equilibrium point of the neural network is consistent with the optimal solution of the original optimization problem. The existence and uniqueness of the equilibrium point of the neural network are proved. Application inequality technique proved global exponential stability of the network. Some numerical examples are given …
The paper is devoted to an analysis of a new constraint qualification and a derivation of the strongest existing optimality conditions for nonsmooth mathematical programming problems with equality and inequality constraints in terms of Demyanov-Rubinov-Polyakova quasidifferentials under the minimal possible assumptions. To this end, we obtain a novel description of convex subcones of the contingent cone to a set defined by quasidifferentiable equality and inequality constraints with the use of a new constraint qualification. We utilize these description and constraint qualification to derive t…
We consider average-cost Markov decision processes (MDPs) with Borel state and action spaces and universally measurable policies. For the nonnegative cost model and an unbounded cost model with a Lyapunov-type stability character, we introduce a set of new conditions under which we prove the average cost optimality inequality (ACOI) via the vanishing discount factor approach. Unlike most existing results on the ACOI, our result does not require any compactness and continuity conditions on the MDPs. Instead, the main idea is to use the almost-uniform-convergence property of a pointwise converge…
We generalize the successive continuation paradigm introduced by Kernévez and Doedel [16] for locating locally optimal solutions of constrained optimization problems to the case of simultaneous equality and inequality constraints. The analysis shows that potential optima may be found at the end of a sequence of easily-initialized separate stages of continuation, without the need to seed the first stage of continuation with nonzero values for the corresponding Lagrange multipliers. A key enabler of the proposed generalization is the use of complementarity functions to define relaxed complementa…
We provide a theoretical algorithm for checking local optimality and escaping saddles at nondifferentiable points of empirical risks of two-layer ReLU networks. Our algorithm receives any parameter value and returns: local minimum, second-order stationary point, or a strict descent direction. The presence of $M$ data points on the nondifferentiability of the ReLU divides the parameter space into at most $2^M$ regions, which makes analysis difficult. By exploiting polyhedral geometry, we reduce the total computation down to one convex quadratic program (QP) for each hidden node, $O(M)$ (in)equa…
