We consider the differentiation of the value function for parametric optimization problems. Such problems are ubiquitous in Machine Learning applications such as structured support vector machines, matrix factorization and min-min or minimax problems in general. Existing approaches for computing the derivative rely on strong assumptions of the parametric function. Therefore, in several scenarios there is no theoretical evidence that a given algorithmic differentiation strategy computes the true gradient information of the value function. We leverage a well known result from convex duality theory to relax the conditions and to derive convergence rates of the derivative approximation for several classes of parametric optimization problems in Machine Learning. We demonstrate the versatility of our approach in several experiments, including non-smooth parametric functions. Even in settings where other approaches are applicable, our duality based strategy shows a favorable performance.
We show how to efficiently compute the derivative (when it exists) of the solution map of log-log convex programs (LLCPs). These are nonconvex, nonsmooth optimization problems with positive variables that become convex when the variables, objective functions, and constraint functions are replaced with their logs. We focus specifically on LLCPs generated by disciplined geometric programming, a grammar consisting of a set of atomic functions with known log-log curvature and a composition rule for combining them. We represent a parametrized LLCP as the composition of a smooth transformation of pa…
We study conjugate and Lagrange dualities for composite optimization problems within the framework of abstract convexity. We provide conditions for zero duality gap in conjugate duality. For Lagrange duality, intersection property is applied to obtain zero duality gap. Connection between Lagrange dual and conjugate dual is also established. Examples related to convex and weakly convex functions are given.
We investigate Lagrangian duality for nonconvex optimization problems. To this aim we use the $Φ$-convexity theory and minimax theorem for $Φ$-convex functions. We provide conditions for zero duality gap and strong duality. Among the classes of functions, to which our duality results can be applied, are prox-bounded functions, DC functions, weakly convex functions and paraconvex functions.
We consider the problem of efficiently computing the derivative of the solution map of a convex cone program, when it exists. We do this by implicitly differentiating the residual map for its homogeneous self-dual embedding, and solving the linear systems of equations required using an iterative method. This allows us to efficiently compute the derivative operator, and its adjoint, evaluated at a vector. These correspond to computing an approximate new solution, given a perturbation to the cone program coefficients (i.e., perturbation analysis), and to computing the gradient of a function of t…
We associate with each convex optimization problem posed on some locally convex space with an infinite index set T, and a given non-empty family H formed by finite subsets of T, a suitable Lagrangian-Haar dual problem. We provide reverse H-strong duality theorems, H-Farkas type lemmas and optimality theorems. Special attention is addressed to infinite and semi-infinite linear optimization problems.
This paper provides an unique dual representation of set-valued lower semi-continuous quasiconvex and convex functions. The results are based on a duality result for increasing set valued functions.
Geometric duality theory for multiple objective linear programming problems turned out to be very useful for the development of efficient algorithms to generate or approximate the whole set of nondominated points in the outcome space. This article extends the geometric duality theory to convex vector optimization problems.
We associate with each convex optimization problem, posed on some locally convex space, with infinitely many constraints indexed by the set T, and a given non-empty family H of finite subsets of T, a suitable Lagrangian-Haar dual problem. We obtain necessary and sufficient conditions for H-reducibility, that is, equivalence to some subproblem obtained by replacing the whole index set T by some element of H. Special attention is addressed to linear optimization, infinite and semi-infinite, and to convex problems with a countable family of constraints. Results on zero H-duality gap and on H-(sta…
In this article we develop a duality principle suitable for a large class of problems in optimization. The main result is obtained through basic tools of convex analysis and duality theory. We establish a correct relation between the critical points of the primal and dual formulations and formally prove there is no duality gap between such formulations, in a local extremal context.
This paper provides a novel definition for Lyapunov functions for difference inclusions defined by convex processes. It is shown that this definition reflects stability properties of nonstrict convex processes better than previously used definitions. In addition the paper presents conditions under which a weak Lyapunov function for a convex process yields a strong Lyapunov function for the dual of the convex process.
This note establishes a limiting formula for the conic Lagrangian dual of a convex infinite optimization problem, correcting the classical version of Karney [Math. Programming 27 (1983) 75-82] for convex semi-infinite programs. A reformulation of the convex infinite optimization problem with a single constraint leads to a limiting formula for the corresponding Lagrangian dual, called sup-dual, and also for the primal problem in the case when strong Slater condition holds, which also entails strong sup-duality.
The aim of this paper is to implement some new techniques, based on conjugate duality in convex optimization, for proving the existence of global error bounds for convex inequality systems. We deal first of all with systems described via one convex inequality and extend the achieved results, by making use of a celebrated scalarization function, to convex inequality systems expressed by means of a general vector function. We also propose a second approach for guaranteeing the existence of global error bounds of the latter, which meanwhile sharpens the classical result of Robinson.
We extend in two ways the standard Karush-Kuhn-Tucker optimality conditions to problems with a convex objective, convex functional constraints, and the extra requirement that some of the variables must be integral. While the standard Karush-Kuhn-Tucker conditions involve separating hyperplanes, our extension is based on lattice-free polyhedra. Our optimality conditions allow us to define an exact dual of our original mixed-integer convex problem.
We present a new duality theory for non-convex variational problems, under possibly mixed Dirichlet and Neumann boundary conditions. The dual problem reads nicely as a linear programming problem, and our main result states that there is no duality gap. Further, we provide necessary and sufficient optimality conditions, and we show that our duality principle can be reformulated as a min-max result which is quite useful for numerical implementations. As an example, we illustrate the application of our method to a celebrated free boundary problem. The results were announced in \cite{BoFr}.
In this paper we introduce a new dual program, which is representable as a semi-definite linear programming problem, for a primal convex minimax programming model problem and show that there is no duality gap between the primal and the dual whenever the functions involved are SOS-convex polynomials. Under a suitable constraint qualification, we derive strong duality results for this class of minimax problems. Consequently, we present applications of our results to robust SOS-convex programming problems under data uncertainty and to minimax fractional programming problems with SOS-convex polyno…
We present new results on optimization problems where the involved functions are evenly convex. By means of a generalized conjugation scheme and the perturbation theory introduced by Rockafellar, we propose an alternative dual problem for a general optimization one defined on a separated locally convex topological space. Sufficient conditions for converse and total duality involving the even convexity of the perturbation function and $c$-subdifferentials are given. Formulae for the $c$-subdifferential and biconjugate of the objective function of a general optimization problem are provided, too…
This paper addresses the study and applications of polyhedral duality of locally convex topological vector (LCTV) spaces. We first revisit the classical Rockafellar's proper separation theorem for two convex sets one which is polyhedral and then present its LCTV extension with replacing the relative interior by its quasi-relative interior counterpart. Then we apply this result to derive enhanced calculus rules for normals to convex sets, coderivatives of convex set-valued mappings, and subgradients of extended-real-valued functions under certain polyhedrality requirements in LCTV spaces by dev…
By applying the perturbation function approach, we propose the Lagrangian and the conjugate duals for minimization problems of the sum of two, generally nonconvex, functions. The main tools are the $Φ$-convexity theory and minimax theorems for $Φ$-convex functions. We provide conditions ensuring zero duality gap and introduce $Φ$-Karush-Kuhn-Tucker conditions that characterize solutions to primal and dual problems. We also discuss the relationship between the dual problems introduced in the present investigation and some conjugate-type duals existing in the literature.
Differentiating Nonsmooth Solutions to Parametric Monotone Inclusion Problems
Toulouse School of Economics（图卢兹经济学院）；University of Toulouse（图卢兹大学）；IRIT（图卢兹计算机科学研究所）；CNRS（法国国家科学研究中心）；Université Toulouse III Paul Sabatier（图卢兹第三大学保罗·萨巴蒂埃大学）；Institut Universitaire de France（法兰西大学研究院）；CVN（认知与神经计算研究中心）；CentraleSupélec（巴黎中央理工-高等电力学院）；Université Paris-Saclay（巴黎萨克雷大学）
This article develops a duality principle applicable to a large class of variational problems. Firstly, we apply the results to a Ginzburg-Landau type model. In a second step, we develop another duality principle and related primal dual variational formulation and such an approach includes optimality conditions which guarantee zero duality gap between the primal and dual formulations. We emphasize in both cases the dual variational formulations obtained have large regions of concavity about the critical points in question.
This article develops a duality principle for a class of optimization problems in $\mathbb{R}^n$. The results are obtained based on standard tools of convex analysis and on a well known result of Toland for D.C. optimization. Global sufficient optimality conditions are also presented as well as relations between the critical points of the primal and dual formulations. Finally we formally prove there is no duality gap between the primal and dual formulations in a local extremal context.
This article develops duality principles applicable to non-convex models in the calculus of variations. The results here developed are applied to Ginzburg-Landau type equations. For the first and second duality principles, through an optimality criterion developed for the dual formulations, we qualitatively classify the critical points of the primal and dual functionals in question. We formally prove there is no duality gap between the primal and dual formulations in a local extremal context. Finally, in the last sections, we present a global existence result, a duality principle and respectiv…
This paper studies convex problems of Bolza in the conjugate duality framework of Rockafellar. We parameterize the problem by a general Borel measure which has direct economic interpretation in problems of financial economics. We derive a dual representation for the optimal value function in terms of continuous dual arcs and we give conditions for the existence of solutions. Combined with well-known results on problems of Bolza over absolutely continuous arcs, we obtain optimality conditions in terms of extended Hamiltonian conditions.
We are interested in solving convex optimization problems with large numbers of constraints. Randomized algorithms, such as random constraint sampling, have been very successful in giving nearly optimal solutions to such problems. In this paper, we combine random constraint sampling with the classical primal-dual algorithm for convex optimization problems with large numbers of constraints, and we give a convergence rate analysis. We then report numerical experiments that verify the effectiveness of this algorithm.
Fourier-Motzkin elimination is a projection algorithm for solving finite linear programs. We extend Fourier-Motzkin elimination to semi-infinite linear programs which are linear programs with finitely many variables and infinitely many constraints. Applying projection leads to new characterizations of important properties for primal-dual pairs of semi-infinite programs such as zero duality gap, feasibility, boundedness, and solvability. Extending the Fourier-Motzkin elimination procedure to semi-infinite linear programs yields a new classification of variables that is used to determine the exi…
This article develops duality principles applicable to the non-linear Kirchhoff-Love model of plates. The results are obtained through standard tools of convex analysis, functional analysis, calculus of variations and duality theory. The main duality principle concerns a convex (in fact concave) dual variational formulation and related new optimality conditions for the model in question. Finally, in the last section we develop some global existence results for a similar model in elasticity.
Bilevel programs are optimization problems where some variables are solutions to optimization problems themselves, and they arise in a variety of control applications, including: control of vehicle traffic networks, inverse reinforcement learning and inverse optimization, and robust control for human-automation systems. This paper develops a duality-based approach to solving bilevel programs where the lower level problem is convex. Our approach is to use partial dualization to construct a new dual function that is differentiable, unlike the Lagrangian dual that is only directionally differenti…
In this paper we report further progress towards a complete theory of state-independent expected utility maximization with semimartingale price processes for arbitrary utility function. Without any technical assumptions we establish a surprising Fenchel duality result on conjugate Orlicz spaces, offering a new economic insight into the nature of primal optima and providing fresh perspective on the classical papers of Kramkov and Schachermayer (1999, 2003). The analysis points to an intriguing interplay between no-arbitrage conditions and standard convex optimization and motivates study of the …
The paper deals with the optimal control problem described by second order evolution differential inclusions; to this end first we use an auxiliary problem with second order discrete and discrete-approximate inclusions. Then applying infimal convolution concept of convex functions, step by step we construct the dual problems for discrete, discrete-approximate and differential inclusions and prove duality results. It seems that the Euler-Lagrange type inclusions are "duality relations" for both primary and dual problems and that the dual problem for discrete-approximate problem make a bridge be…
This short communication develops a convex dual variational formulation for a large class of models in variational optimization. The results are established through basic tools of functional analysis, convex analysis and duality theory. The main duality principle is developed as an application to a Ginzburg-Landau type system in superconductivity in the absence of a magnetic field.
We study geometric duality for convex vector optimization problems. For a primal problem with a $q$-dimensional objective space, we formulate a dual problem with a $(q+1)$-dimensional objective space. Consequently, different from an existing approach, the geometric dual problem does not depend on a fixed direction parameter and the resulting dual image is a convex cone. We prove a one-to-one correspondence between certain faces of the primal and dual images. In addition, we show that a polyhedral approximation for one image gives rise to a polyhedral approximation for the other. Based on this,…
Generalized polyhedral convex optimization problems in locally convex Hausdorff topological vector spaces are studied systematically in this paper. We establish solution existence theorems, necessary and sufficient optimality conditions, weak and strong duality theorems. In particular, we show that the dual problem has the same structure as the primal problem, and the strong duality relation holds under three different sets of conditions.
In this paper, we show that the subadditive dual of a feasible conic mixed-integer program (MIP) is a strong dual whenever it is feasible. Moreover, we show that this dual feasibility condition is equivalent to feasibility of the conic dual of the continuous relaxation of the conic MIP. In addition, we prove that all known conditions and other 'natural' conditions for strong duality, such as strict mixed-integer feasibility, boundedness of the feasible set or essentially strict feasibility imply that the subadditive dual is feasible. As an intermediate result, we extend the so-called 'finitene…
Level-set methods for convex optimization are predicated on the idea that certain problems can be parameterized so that their solutions can be recovered as the limiting process of a root-finding procedure. This idea emerges time and again across a range of algorithms for convex problems. Here we demonstrate that strong duality is a necessary condition for the level-set approach to succeed. In the absence of strong duality, the level-set method identifies $ε$-infeasible points that do not converge to a feasible point as $ε$ tends to zero. The level-set approach is also used as a proof technique…
We study a special class of non-convex functions which appear in nonlinear elasticity; and we prove that they have well-defined Legandre transforms. Several examples are given, and an application to a nonlinear eigenvalue problem
This paper proposes a general duality framework for the problem of minimizing a convex integral functional over a space of stochastic processes adapted to a given filtration. The framework unifies many well-known duality frameworks from operations research and mathematical finance. The unification allows the extension of some useful techniques from these two fields to a much wider class of problems. In particular, combining certain finite-dimensional techniques from convex analysis with measure theoretic techniques from mathematical finance, we are able to close the duality gap in some situati…
We propose to solve large instances of the non-convex optimization problems reformulated with canonical duality theory. To this aim we propose an interior point potential reduction algorithm based on the solution of the primal-dual total complementarity (Lagrange) function. We establish the global convergence result for the algorithm under mild assumptions and demonstrate the method on instances of the Sensor Network Localization problem. Our numerical results are promising and show the possibility of devising efficient interior points methods for non-convex duality.
We consider the problem of minimizing a sum of non-convex functions over a compact domain, subject to linear inequality and equality constraints. Approximate solutions can be found by solving a convexified version of the problem, in which each function in the objective is replaced by its convex envelope. We propose a randomized algorithm to solve the convexified problem which finds an $ε$-suboptimal solution to the original problem. With probability one, $ε$ is bounded by a term proportional to the maximal number of active constraints in the problem. The bound does not depend on the number of …
In this paper, we introduce new properties of the relative interior calculus for nearly convex sets, functions, and set-valued mappings. These properties are important for the development of duality theory in optimization. Then we investigate optimal value functions defined by nearly convex functions and nearly convex set-valued mappings, and derive the near convexity of the optimal value function under a qualification condition. We also develop formulas for subgradients and Fenchel conjugates of this class of functions, and explore their applications to duality theory.
This article develops a primal dual formulation for a primal proximal approach suitable for a large class of non-convex models in the calculus of variations. The results are established through standard tools of functional analysis, convex analysis and duality theory and are applied to a Ginzburg-Landau type model. Finally, in the last two sections, we present concerning optimal conditions and another related duality principle for the model in question.
