We investigate Lagrangian duality for nonconvex optimization problems. To this aim we use the $Φ$-convexity theory and minimax theorem for $Φ$-convex functions. We provide conditions for zero duality gap and strong duality. Among the classes of functions, to which our duality results can be applied, are prox-bounded functions, DC functions, weakly convex functions and paraconvex functions.
This paper presents a canonical duality theory for solving a general nonconvex constrained optimization problem within a unified framework to cover Lagrange multiplier method and KKT theory. It is proved that if both target function and constraints possess certain patterns necessary for modeling real systems, a perfect dual problem (without duality gap)can be obtained in a unified form with global optimality conditions provided. While the popular augmented Lagrangian method may produce more difficult nonconvex problems due to the nonlinearity of constraints.
We study nonconvex quadratic problems (QPs) with quadratic separable constraints, where these constraints can be defined both as inequalities or equalities. We derive sufficient conditions for these types of problems to present the S-property, which ultimately guarantees strong duality between the primal and dual problems of the QP. We study the existence of solutions and propose a novel distributed algorithm to solve the problem optimally when the S-property is satisfied. Finally, we illustrate our theoretical results proving that the robust least squares problem with multiple constraints has…
By applying the perturbation function approach, we propose the Lagrangian and the conjugate duals for minimization problems of the sum of two, generally nonconvex, functions. The main tools are the $Φ$-convexity theory and minimax theorems for $Φ$-convex functions. We provide conditions ensuring zero duality gap and introduce $Φ$-Karush-Kuhn-Tucker conditions that characterize solutions to primal and dual problems. We also discuss the relationship between the dual problems introduced in the present investigation and some conjugate-type duals existing in the literature.
Based on concepts like kth convex hull and finer characterization of nonconvexity of a function, we propose a refinement of the Shapley-Folkman lemma and derive a new estimate for the duality gap of nonconvex optimization problems with separable objective functions. We apply our result to a network flow problem and the dynamic spectrum management problem in communication systems as examples to demonstrate that the new bound can be qualitatively tighter than the existing ones. The idea is also applicable to cases with general nonconvex constraints.
We consider an optimization problem with positively homogeneous functions in its objective and constraint functions. Examples of such positively homogeneous functions include the absolute value function and the $p$-norm function, where $p$ is a positive real number. The problem, which is not necessarily convex, extends the absolute value optimization proposed in [O. L. Mangasarian, Absolute value programming, Computational Optimization and Applications 36 (2007) pp. 43-53]. In this work, we propose a dual formulation that, differently from the Lagrangian dual approach, has a closed-form and so…
General nonconvex optimization problems are studied by using the canonical duality-triality theory. The triality theory is proved for sums of exponentials and quartic polynomials, which solved an open problem left in 2003. This theory can be used to find the global minimum and local extrema, which bridges a gap between global optimization and nonconvex mechanics. Detailed applications are illustrated by several examples.
This paper revisits the well-studied fixed point problem from a unified viewpoint of mathematical modeling and canonical duality theory, i.e. the original problem is first reformulated as a nonconvex optimization problem, its well-posedness is discussed based on objectivity principle in continuum physics; then the canonical duality theory is applied for solving this problem to obtain not only all fixed points, but also their stability properties. Applications are illustrated by challenging problems governed by nonconvex polynomial, exponential, and logarithmic operators. This paper shows that …
This paper presents a canonical dual approach for solving nonconvex quadratic minimization problem. By using the canonical duality theory, nonconvex primal minimization problems over n-dimensional Lorentz cone can be transformed into certain canonical dual problems with only one dual variable, which can be solved by using standard convex minimization methods. Extremality conditions of these solutions are classified by the triality theory. Applications are illustrated.
We study conjugate and Lagrange dualities for composite optimization problems within the framework of abstract convexity. We provide conditions for zero duality gap in conjugate duality. For Lagrange duality, intersection property is applied to obtain zero duality gap. Connection between Lagrange dual and conjugate dual is also established. Examples related to convex and weakly convex functions are given.
This paper presents a canonical dual approach for solving a nonconvex global optimization problem governed by a sum of fourth-order polynomial and a log-sum-exp function. Such a problem arises extensively in engineering and sciences. Based on the canonical duality-triality theory, this nonconvex problem is transformed to an equivalent dual problem, which can be solved easily under certain conditions. We proved that both global minimizer and the biggest local extrema of the primal problem can be obtained analytically from the canonical dual solutions. As two special cases, a quartic polynomial …
This paper presents a set of complete solutions of a nonconvex variational problem with a double-well potential. Based on the canonical duality-triality theory, the associated nonlinear differential equation with either Dirichlet/Neumann or mixed boundary conditions can be converted into an algebraic equation, which can be solved analytically to obtain all solutions in the dual space. Both global and local extremality conditions are identified by the triality theory. In the application part, typical mechanical models with specificsources and boundary conditions in $\mathbb{R}^2$ are exhibited.
The paper introduces several new concepts for solving nonconvex or nonsmooth optimization problems, including convertible nonconvex function, exact convertible nonconvex function and differentiable convertible nonconvex function. It is proved herein many nonconvex functions or nonsmooth (or discontinuous) functions are actually convertible nonconvex functions and convertible nonconvex function operations such as addition, subtraction, multiplication or division result in convertible nonconvex functions. The sufficient condition for judging a global optimal solution to unconstrained optimizatio…
We study a canonical duality method to solve a mixed-integer nonconvex fourth-order polynomial minimization problem with fixed cost terms. This constrained nonconvex problem can be transformed into a continuous concave maximization dual problem without duality gap. The global optimality conditions are proposed and the existence and uniqueness criteria are discussed. Application to a decoupled mixed-integer problem is illustrated and analytic solution for a global minimum is obtained under some suitable conditions. Several examples are given to show the method is effective.
This paper presents a canonical dual approach to the problem of minimizing the sum of a quadratic function and the ratio of nonconvex function and quadratic functions, which is a type of non-convex optimization problem subject to an elliptic constraint. We first relax the fractional structure by introducing a family of parametric subproblems. Under certain conditions, we show that the canonical dual of each subproblem becomes a two-dimensional concave maximization problem that exhibits no duality gap. Since the infimum of the optima of the parameterized subproblems leads to a solution to the o…
In this paper, we perform sensitivity analysis for the maximal value function which is the optimal value function for a parametric maximization problem. Our aim is to study various subdifferentials for the maximal value function. We obtain upper estimates of Fréchet, limiting, and horizon subdifferentials of the maximal value function by using some sensitivity analysis techniques sophisticatedly. The derived upper estimates depend only on the union of all solutions and not on its convex hull or only one solution from the solution set. Finally, we apply the derived results to develop some new n…
The main problem considered in the present paper is to single out classes of convex sets, whose convexity property is preserved under nonlinear smooth transformations. Extending an approach due to B.T. Polyak, the present study focusses on the class of uniformly convex subsets of Banach spaces. As a main result, a quantitative condition linking the modulus of convexity of such kind of set, the regularity behaviour around a point of a nonlinear mapping and the Lipschitz continuity of its derivative is established, which ensures the images of uniformly convex sets to remain uniformly convex. App…
Although originally designed and analyzed for convex problems, the alternating direction method of multipliers (ADMM) and its close relatives, Douglas-Rachford splitting (DRS) and Peaceman-Rachford splitting (PRS), have been observed to perform remarkably well when applied to certain classes of structured nonconvex optimization problems. However, partial global convergence results in the nonconvex setting have only recently emerged. In this paper we show how the Douglas-Rachford envelope (DRE), introduced in 2014, can be employed to unify and considerably simplify the theory for devising globa…
This paper presents a canonical duality theory for solving nonconvex minimization problem of Rosenbrock function. Extensive numerical results show that this benchmark test problem can be solved precisely and efficiently to obtain global optimal solutions.
We study a special class of non-convex quadratic programs subject to two (possibly indefinite) quadratic constraints when the level sets of the constraint functions are {\it not} arranged {\it alternatively.} It is shown in the paper that this class of problems admit strong duality following a tight SDP relaxation, without assuming primal or dual Slater conditions. Our results cover Ye and Zhang's development in 2003 and the generalized trust region subproblems (GTRS) as special cases. Through the novel geometric view and some simple examples, we can explain why the problem becomes very hard w…
We consider the problem of finding a low rank symmetric matrix satisfying a system of linear equations, as appears in phase retrieval. In particular, we solve the gauge dual formulation, but use a fast approximation of the spectral computations to achieve a noisy solution estimate. This estimate is then used as the initialization of an alternating gradient descent scheme over a nonconvex rank-1 matrix factorization formulation. Numerical results on small problems show consistent recovery, with very low computational cost.
This paper proposes and develops new linesearch methods with inexact gradient information for finding stationary points of nonconvex continuously differentiable functions on finite-dimensional spaces. Some abstract convergence results for a broad class of linesearch methods are stablished. A general scheme for inexact reduced gradient (IRG) methods is proposed, where the errors in the gradient approximation automatically adapt with the magnitudes of the exact gradients. The sequences of iterations are shown to obtain stationary accumulation points when different stepsize selections are employe…
This paper presents a canonical d.c. (difference of canonical and convex functions) programming problem, which can be used to model general global optimization problems in complex systems. It shows that by using the canonical duality theory, a large class of nonconvex minimization problems can be equivalently converted to a unified concave maximization problem over a convex domain, which can be solved easily under certain conditions. Additionally, a detailed proof for triality theory is provided, which can be used to identify local extremal solutions. Applications are illustrated and open prob…
Using tools provided by the theory of abstract convexity, we extend conditions for zero duality gap to the context of nonconvex and nonsmooth optimization. Mimicking the classical setting, an abstract convex function is the upper envelope of a family of abstract affine functions (being conventional vertical translations of the abstract linear functions). We establish new conditions for zero duality gap under no topological assumptions on the space of abstract linear functions. In particular, we prove that the zero duality gap property can be fully characterized in terms of an inclusion involvi…
The goal of this paper is to solve a class of high-order polynomial benchmark optimization problems, including the Goldstein-Price problem and the Three Hump Camel Back problem. By using a generalized canonical duality theory, we are able to transform the nonconvex primal problems to concave dual problems over convex domain(without duality gap), which can be solved easily to obtain global solutions.
This paper aims to develop distributed algorithms for nonconvex optimization problems with complicated constraints associated with a network. The network can be a physical one, such as an electric power network, where the constraints are nonlinear power flow equations, or an abstract one that represents constraint couplings between decision variables of different agents. Despite the recent development of distributed algorithms for nonconvex programs, highly complicated constraints still pose a significant challenge in theory and practice. We first identify some difficulties with the existing a…
We propose a trust-region type method for a class of nonsmooth nonconvex optimization problems where the objective function is a summation of a (probably nonconvex) smooth function and a (probably nonsmooth) convex function. The model function of our trust-region subproblem is always quadratic and the linear term of the model is generated using abstract descent directions. Therefore, the trust-region subproblems can be easily constructed as well as efficiently solved by cheap and standard methods. When the accuracy of the model function at the solution of the subproblem is not sufficient, we a…
A unified model is addressed for general optimization problems in multi-scale complex systems. Based on necessary conditions and basic principles in physics, the canonical duality-triality theory is presented in a precise way to include traditional duality theories and popular methods as special applications. Two conjectures on NP-hardness are discussed, which should play important roles for correctly understanding and efficiently solving challenging real-world problems. Applications are illustrated for both nonconvex continuous optimization and mixed integer nonlinear programming. Misundersta…
The first part of this work established the foundations of a radial duality between nonnegative optimization problems, inspired by the work of (Renegar, 2016). Here we utilize our radial duality theory to design and analyze projection-free optimization algorithms that operate by solving a radially dual problem. In particular, we consider radial subgradient, smoothing, and accelerated methods that are capable of solving a range of constrained convex and nonconvex optimization problems and that can scale-up more efficiently than their classic counterparts. These algorithms enjoy the same benefit…
We study the convergence of general abstract descent methods applied to a lower semicontinuous nonconvex function f that satisfies the Kurdyka-Lojasiewicz inequality in a Hilbert space. We prove that any precompact sequence converges to a critical point of f and obtain new convergence rates both for the values and the iterates. The analysis covers alternating versions of the forward-backward method with variable metric and relative errors. As an example, a nonsmooth and nonconvex version of the Levenberg-Marquardt algorithm is detailled.
Douglas--Rachford for multioperator comonotone inclusions with applications to multiblock optimization
Center for Advanced Intelligence Project, RIKEN（RIKEN先进智能项目中心）；School of Science, RMIT University（皇家墨尔本理工大学理学院）；Department of Mathematical Informatics, Graduate School of Information Science and Technology, University of Tokyo（东京大学信息理工学研究科数学信息学专业）
The uniform quadratic optimizatin problem (UQ) is a nonconvex quadratic constrained quadratic programming (QCQP) sharing the same Hessian matrix. Based on the second-order cone programming (SOCP) relaxation, we establish a new sufficient condition to guarantee strong duality for (UQ) and then extend it to (QCQP), which not only covers several well-known results in literature but also partially gives answers to a few open questions. For convex constrained nonconvex (UQ), we propose an improved approximation algorithm based on (SOCP). Our approximation bound is dimensional independent. As an app…
In this article we utilise abstract convexity theory in order to unify and generalize many different concepts from nonsmooth analysis. We introduce the concepts of abstract codifferentiability, abstract quasidifferentiability and abstract convex (concave) approximations of a nonsmooth function mapping a topological vector space to an order complete topological vector lattice. We study basic properties of these notions, construct elaborate calculus of abstract codifferentiable functions and discuss continuity of abstract codifferential. We demonstrate that many classical concepts of nonsmooth a…
This paper demonstrates a mathematically correct and computationally powerful method for solving 3D topology optimization problems. This method is based on canonical duality theory (CDT) developed by Gao in nonconvex mechanics and global optimization. It shows that the so-called NP-hard knapsack problem in topology optimization can be solved deterministically in polynomial time via a canonical penalty-duality (CPD) method to obtain precise 0-1 global optimal solution at each volume evolution. The relation between this CPD method and Gao's pure complementary energy principle is revealed for the…
In this paper, we consider the canonical water network design problem, which contains nonconvex potential loss functions and discrete resistance choices with varying costs. Traditionally, to resolve the nonconvexities of this problem, relaxations of the potential loss constraints have been applied to yield a more tractable mixed-integer convex program (MICP). However, design solutions to these relaxed problems may not be feasible with respect to the full nonconvex physics. In this paper, it is shown that, in fact, the original mixed-integer nonconvex program can be reformulated exactly as an M…
We propose a duality scheme for solving constrained nonsmooth and nonconvex optimization problems in a reflexive Banach space. We establish strong duality for a very general type of augmented Lagrangian, in which we assume a less restrictive type of coercivity on the augmenting function. We solve the dual problem (in a Hilbert space) using a deflected subgradient method via this general augmented Lagrangian. We provide two choices of step-size for the method. For both choices, we prove that every weak accumulation point of the primal sequence is a primal solution. We also prove strong converge…
We consider the Monge-Kantorovich transport problem in an abstract measure theoretic setting. Our main result states that duality holds if $c:X\times Y\to [0,\infty)$ is an arbitrary Borel measurable cost function on the product of Polish spaces $X,Y$. In the course of the proof we show how to relate a non - optimal transport plan to the optimal transport costs via a ``subsidy'' function and how to identify the dual optimizer. We also provide some examples showing the limitations of the duality relations.
In recent work arXiv:2109.07820 we have shown the equivalence of the widely used nonconvex (generalized) branched transport problem with a shape optimization problem of a street or railroad network, known as (generalized) urban planning problem. The argument was solely based on an explicit construction and characterization of competitors. In the current article we instead analyse the dual perspective associated with both problems. In more detail, the shape optimization problem involves the Wasserstein distance between two measures with respect to a metric depending on the street network. We sh…
The canonical duality theory has provided with a unified analytic solution to a range of discrete and continuous problems in global optimization, which can transform a nonconvex primal problem to a concave maximization dual problem over a convex domain without duality gap. This paper shows that under certain conditions, this canonical dual problem is equivalent to the standard semi-definite programming (SDP) problem, which can be solved by well-developed software packages. In order to avoid certain difficulties of using the SDP method, four strategies are proposed based on unconstrained approa…
