This paper presents a canonical duality theory for solving a general nonconvex constrained optimization problem within a unified framework to cover Lagrange multiplier method and KKT theory. It is proved that if both target function and constraints possess certain patterns necessary for modeling real systems, a perfect dual problem (without duality gap)can be obtained in a unified form with global optimality conditions provided. While the popular augmented Lagrangian method may produce more difficult nonconvex problems due to the nonlinearity of constraints.
We investigate Lagrangian duality for nonconvex optimization problems. To this aim we use the $Φ$-convexity theory and minimax theorem for $Φ$-convex functions. We provide conditions for zero duality gap and strong duality. Among the classes of functions, to which our duality results can be applied, are prox-bounded functions, DC functions, weakly convex functions and paraconvex functions.
We study nonconvex quadratic problems (QPs) with quadratic separable constraints, where these constraints can be defined both as inequalities or equalities. We derive sufficient conditions for these types of problems to present the S-property, which ultimately guarantees strong duality between the primal and dual problems of the QP. We study the existence of solutions and propose a novel distributed algorithm to solve the problem optimally when the S-property is satisfied. Finally, we illustrate our theoretical results proving that the robust least squares problem with multiple constraints has…
By applying the perturbation function approach, we propose the Lagrangian and the conjugate duals for minimization problems of the sum of two, generally nonconvex, functions. The main tools are the $Φ$-convexity theory and minimax theorems for $Φ$-convex functions. We provide conditions ensuring zero duality gap and introduce $Φ$-Karush-Kuhn-Tucker conditions that characterize solutions to primal and dual problems. We also discuss the relationship between the dual problems introduced in the present investigation and some conjugate-type duals existing in the literature.
This paper presents a canonical dual approach for solving nonconvex quadratic minimization problem. By using the canonical duality theory, nonconvex primal minimization problems over n-dimensional Lorentz cone can be transformed into certain canonical dual problems with only one dual variable, which can be solved by using standard convex minimization methods. Extremality conditions of these solutions are classified by the triality theory. Applications are illustrated.
Based on concepts like kth convex hull and finer characterization of nonconvexity of a function, we propose a refinement of the Shapley-Folkman lemma and derive a new estimate for the duality gap of nonconvex optimization problems with separable objective functions. We apply our result to a network flow problem and the dynamic spectrum management problem in communication systems as examples to demonstrate that the new bound can be qualitatively tighter than the existing ones. The idea is also applicable to cases with general nonconvex constraints.
We consider an optimization problem with positively homogeneous functions in its objective and constraint functions. Examples of such positively homogeneous functions include the absolute value function and the $p$-norm function, where $p$ is a positive real number. The problem, which is not necessarily convex, extends the absolute value optimization proposed in [O. L. Mangasarian, Absolute value programming, Computational Optimization and Applications 36 (2007) pp. 43-53]. In this work, we propose a dual formulation that, differently from the Lagrangian dual approach, has a closed-form and so…
This paper presents a canonical dual approach for solving a nonconvex global optimization problem governed by a sum of fourth-order polynomial and a log-sum-exp function. Such a problem arises extensively in engineering and sciences. Based on the canonical duality-triality theory, this nonconvex problem is transformed to an equivalent dual problem, which can be solved easily under certain conditions. We proved that both global minimizer and the biggest local extrema of the primal problem can be obtained analytically from the canonical dual solutions. As two special cases, a quartic polynomial …
General nonconvex optimization problems are studied by using the canonical duality-triality theory. The triality theory is proved for sums of exponentials and quartic polynomials, which solved an open problem left in 2003. This theory can be used to find the global minimum and local extrema, which bridges a gap between global optimization and nonconvex mechanics. Detailed applications are illustrated by several examples.
This paper revisits the well-studied fixed point problem from a unified viewpoint of mathematical modeling and canonical duality theory, i.e. the original problem is first reformulated as a nonconvex optimization problem, its well-posedness is discussed based on objectivity principle in continuum physics; then the canonical duality theory is applied for solving this problem to obtain not only all fixed points, but also their stability properties. Applications are illustrated by challenging problems governed by nonconvex polynomial, exponential, and logarithmic operators. This paper shows that …
This paper presents a canonical dual approach to the problem of minimizing the sum of a quadratic function and the ratio of nonconvex function and quadratic functions, which is a type of non-convex optimization problem subject to an elliptic constraint. We first relax the fractional structure by introducing a family of parametric subproblems. Under certain conditions, we show that the canonical dual of each subproblem becomes a two-dimensional concave maximization problem that exhibits no duality gap. Since the infimum of the optima of the parameterized subproblems leads to a solution to the o…
We study a canonical duality method to solve a mixed-integer nonconvex fourth-order polynomial minimization problem with fixed cost terms. This constrained nonconvex problem can be transformed into a continuous concave maximization dual problem without duality gap. The global optimality conditions are proposed and the existence and uniqueness criteria are discussed. Application to a decoupled mixed-integer problem is illustrated and analytic solution for a global minimum is obtained under some suitable conditions. Several examples are given to show the method is effective.
In this short note we show, providing counterexamples, that the "two important theorems" in the recent paper [Y, Yuan, Global optimization solutions to a class of non-convex quadratic minimization problems with quadratic constraints, in Canonical Duality Theory, D.Y. Gao et al.\ (eds), (AMMA, volume 37), Springer, 2017] are false.
This paper presents a set of complete solutions of a nonconvex variational problem with a double-well potential. Based on the canonical duality-triality theory, the associated nonlinear differential equation with either Dirichlet/Neumann or mixed boundary conditions can be converted into an algebraic equation, which can be solved analytically to obtain all solutions in the dual space. Both global and local extremality conditions are identified by the triality theory. In the application part, typical mechanical models with specificsources and boundary conditions in $\mathbb{R}^2$ are exhibited.
The paper introduces several new concepts for solving nonconvex or nonsmooth optimization problems, including convertible nonconvex function, exact convertible nonconvex function and differentiable convertible nonconvex function. It is proved herein many nonconvex functions or nonsmooth (or discontinuous) functions are actually convertible nonconvex functions and convertible nonconvex function operations such as addition, subtraction, multiplication or division result in convertible nonconvex functions. The sufficient condition for judging a global optimal solution to unconstrained optimizatio…
This paper presents a new methodology and algorithm for solving post buckling problems of a large deformed elastic beam. The total potential energy of this beam is a nonconvex functional, which can be used to model both pre- and post-buckling phenomena. By using a canonical dual finite element method, a new primal-dual semi-definite programming (PD-SDP) algorithm is presented, which can be used to obtain all possible post-buckled solutions. Applications are illustrated by several numerical examples with different boundary conditions. We find that the global minimum solution of the nonconvex po…
We propose to solve large instances of the non-convex optimization problems reformulated with canonical duality theory. To this aim we propose an interior point potential reduction algorithm based on the solution of the primal-dual total complementarity (Lagrange) function. We establish the global convergence result for the algorithm under mild assumptions and demonstrate the method on instances of the Sensor Network Localization problem. Our numerical results are promising and show the possibility of devising efficient interior points methods for non-convex duality.
This paper presents a canonical d.c. (difference of canonical and convex functions) programming problem, which can be used to model general global optimization problems in complex systems. It shows that by using the canonical duality theory, a large class of nonconvex minimization problems can be equivalently converted to a unified concave maximization problem over a convex domain, which can be solved easily under certain conditions. Additionally, a detailed proof for triality theory is provided, which can be used to identify local extremal solutions. Applications are illustrated and open prob…
This paper presents a canonical duality theory for solving nonconvex minimization problem of Rosenbrock function. Extensive numerical results show that this benchmark test problem can be solved precisely and efficiently to obtain global optimal solutions.
In this paper, we perform sensitivity analysis for the maximal value function which is the optimal value function for a parametric maximization problem. Our aim is to study various subdifferentials for the maximal value function. We obtain upper estimates of Fréchet, limiting, and horizon subdifferentials of the maximal value function by using some sensitivity analysis techniques sophisticatedly. The derived upper estimates depend only on the union of all solutions and not on its convex hull or only one solution from the solution set. Finally, we apply the derived results to develop some new n…
The main problem considered in the present paper is to single out classes of convex sets, whose convexity property is preserved under nonlinear smooth transformations. Extending an approach due to B.T. Polyak, the present study focusses on the class of uniformly convex subsets of Banach spaces. As a main result, a quantitative condition linking the modulus of convexity of such kind of set, the regularity behaviour around a point of a nonlinear mapping and the Lipschitz continuity of its derivative is established, which ensures the images of uniformly convex sets to remain uniformly convex. App…
Although originally designed and analyzed for convex problems, the alternating direction method of multipliers (ADMM) and its close relatives, Douglas-Rachford splitting (DRS) and Peaceman-Rachford splitting (PRS), have been observed to perform remarkably well when applied to certain classes of structured nonconvex optimization problems. However, partial global convergence results in the nonconvex setting have only recently emerged. In this paper we show how the Douglas-Rachford envelope (DRE), introduced in 2014, can be employed to unify and considerably simplify the theory for devising globa…
This paper mainly addresses the Monge mass transfer problem in the 1-D case. Through an ingenious approximation mechanism, one transforms the Monge problem into a sequence of minimization problems, which can be converted into a sequence of nonlinear differential equations with constraints by variational method. The existence and uniqueness of the solution for each equation can be demonstrated by applying the canonical duality method. Moreover, the duality method gives a sequence of perfect dual maximization problems. In the final analysis, one constructs the approximation of optimal mapping fo…
DY Gao solely or together with some of his collaborators applied his Canonical duality theory (CDT) for solving some quadratic optimization problems with quadratic constraints. Unfortunately, in almost all papers we read on CDT there are unclear definitions, non convincing arguments in the proofs, and even false results. The aim of this paper is to treat rigorously quadratic optimization problems by the method suggested by CDT and to compare what we get with the results obtained by DY Gao and his collaborators on this topic in several papers.
We study a special class of non-convex quadratic programs subject to two (possibly indefinite) quadratic constraints when the level sets of the constraint functions are {\it not} arranged {\it alternatively.} It is shown in the paper that this class of problems admit strong duality following a tight SDP relaxation, without assuming primal or dual Slater conditions. Our results cover Ye and Zhang's development in 2003 and the generalized trust region subproblems (GTRS) as special cases. Through the novel geometric view and some simple examples, we can explain why the problem becomes very hard w…
We consider the problem of finding a low rank symmetric matrix satisfying a system of linear equations, as appears in phase retrieval. In particular, we solve the gauge dual formulation, but use a fast approximation of the spectral computations to achieve a noisy solution estimate. This estimate is then used as the initialization of an alternating gradient descent scheme over a nonconvex rank-1 matrix factorization formulation. Numerical results on small problems show consistent recovery, with very low computational cost.
A unified model is addressed for general optimization problems in multi-scale complex systems. Based on necessary conditions and basic principles in physics, the canonical duality-triality theory is presented in a precise way to include traditional duality theories and popular methods as special applications. Two conjectures on NP-hardness are discussed, which should play important roles for correctly understanding and efficiently solving challenging real-world problems. Applications are illustrated for both nonconvex continuous optimization and mixed integer nonlinear programming. Misundersta…
This paper demonstrates a mathematically correct and computationally powerful method for solving 3D topology optimization problems. This method is based on canonical duality theory (CDT) developed by Gao in nonconvex mechanics and global optimization. It shows that the so-called NP-hard knapsack problem in topology optimization can be solved deterministically in polynomial time via a canonical penalty-duality (CPD) method to obtain precise 0-1 global optimal solution at each volume evolution. The relation between this CPD method and Gao's pure complementary energy principle is revealed for the…
The goal of this paper is to solve a class of high-order polynomial benchmark optimization problems, including the Goldstein-Price problem and the Three Hump Camel Back problem. By using a generalized canonical duality theory, we are able to transform the nonconvex primal problems to concave dual problems over convex domain(without duality gap), which can be solved easily to obtain global solutions.
The first part of this work established the foundations of a radial duality between nonnegative optimization problems, inspired by the work of (Renegar, 2016). Here we utilize our radial duality theory to design and analyze projection-free optimization algorithms that operate by solving a radially dual problem. In particular, we consider radial subgradient, smoothing, and accelerated methods that are capable of solving a range of constrained convex and nonconvex optimization problems and that can scale-up more efficiently than their classic counterparts. These algorithms enjoy the same benefit…
The canonical duality theory has provided with a unified analytic solution to a range of discrete and continuous problems in global optimization, which can transform a nonconvex primal problem to a concave maximization dual problem over a convex domain without duality gap. This paper shows that under certain conditions, this canonical dual problem is equivalent to the standard semi-definite programming (SDP) problem, which can be solved by well-developed software packages. In order to avoid certain difficulties of using the SDP method, four strategies are proposed based on unconstrained approa…
Douglas--Rachford for multioperator comonotone inclusions with applications to multiblock optimization
Center for Advanced Intelligence Project, RIKEN（RIKEN先进智能项目中心）；School of Science, RMIT University（皇家墨尔本理工大学理学院）；Department of Mathematical Informatics, Graduate School of Information Science and Technology, University of Tokyo（东京大学信息理工学研究科数学信息学专业）
This paper presents an application of Canonical duality theory to the solution of contact problems with Coulomb friction. The contact problem is formulated as a quasi-variational inequality which solution is found by solving its Karush-Kunt-Tucker system of equations. The complementarity conditions are reformulated by using the Fischer-Burmeister complementarity function, obtaining a non-convex global optimization problem. Then canonical duality theory is applied to reformulate the non-convex global optimization problem and define its optimality conditions, finding a solution of the original q…
The uniform quadratic optimizatin problem (UQ) is a nonconvex quadratic constrained quadratic programming (QCQP) sharing the same Hessian matrix. Based on the second-order cone programming (SOCP) relaxation, we establish a new sufficient condition to guarantee strong duality for (UQ) and then extend it to (QCQP), which not only covers several well-known results in literature but also partially gives answers to a few open questions. For convex constrained nonconvex (UQ), we propose an improved approximation algorithm based on (SOCP). Our approximation bound is dimensional independent. As an app…
Motivated by the problem of online canonical correlation analysis, we propose the \emph{Stochastic Scaled-Gradient Descent} (SSGD) algorithm for minimizing the expectation of a stochastic function over a generic Riemannian manifold. SSGD generalizes the idea of projected stochastic gradient descent and allows the use of scaled stochastic gradients instead of stochastic gradients. In the special case of a spherical constraint, which arises in generalized eigenvector problems, we establish a nonasymptotic finite-sample bound of $\sqrt{1/T}$, and show that this rate is minimax optimal, up to a po…
A novel canonical duality theory (CDT) is presented for solving general bilevel mixed integer nonlinear optimization governed by linear and quadratic knapsack problems. It shows that the challenging knapsack problems can be solved analytically in term of their canonical dual solutions. The existence and uniqueness of these analytical solutions are proved. NP-Hardness of the knapsack problems is discussed. A powerful CDT algorithm combined with an alternative iteration and a volume reduction method is proposed for solving the NP-hard bilevel knapsack problems. Application is illustrated by a be…
This paper presents a canonical duality approach for solving a general topology optimization problem of nonlinear elastic structures. By using finite element method, this most challenging problem can be formulated as a mixed integer nonlinear programming problem (MINLP), i.e. for a given deformation, the first-level optimization is a typical linear constrained 0-1 programming problem, while for a given structure, the second-level optimization is a general nonlinear continuous minimization problem in computational nonlinear elasticity. It is discovered that for linear elastic structures, first-…
Topology optimization for general materials is correctly formulated as a bi-level knapsack problem, which is considered to be NP-hard in global optimization and computer science. By using canonical duality theory (CDT) developed by the author, the linear knapsack problem can be solved analytically to obtain global optimal solution at each design iteration. Both uniqueness, existence, and NP-hardness are discussed. The novel CDT method for general topology optimization is refined and tested by both 2-D and 3-D benchmark problems. Numerical results show that without using filter and any other ar…
This paper presents global optimal solutions to a nonconvex quadratic minimization problem over a sphere constraint. The problem is well-known as a trust region subproblem and has been studied extensively for decades. The main challenge is the so called 'hard case', i.e., the problem has multiple solutions on the boundary of the sphere. By canonical duality theory, this challenging problem is able to reformed as an one-dimensional canonical dual problem without duality gap. Sufficient and necessary conditions are obtained by the triality theory, which can be used to identify whether the proble…
In this paper, we consider the canonical water network design problem, which contains nonconvex potential loss functions and discrete resistance choices with varying costs. Traditionally, to resolve the nonconvexities of this problem, relaxations of the potential loss constraints have been applied to yield a more tractable mixed-integer convex program (MICP). However, design solutions to these relaxed problems may not be feasible with respect to the full nonconvex physics. In this paper, it is shown that, in fact, the original mixed-integer nonconvex program can be reformulated exactly as an M…
We propose a duality scheme for solving constrained nonsmooth and nonconvex optimization problems in a reflexive Banach space. We establish strong duality for a very general type of augmented Lagrangian, in which we assume a less restrictive type of coercivity on the augmenting function. We solve the dual problem (in a Hilbert space) using a deflected subgradient method via this general augmented Lagrangian. We provide two choices of step-size for the method. For both choices, we prove that every weak accumulation point of the primal sequence is a primal solution. We also prove strong converge…
This paper presents a detailed proof of the triality theorem for a class of fourth-order polynomial optimization problems. The method is based on linear algebra but it solves an open problem on the double-min duality left in 2003. Results show that the triality theory holds strongly in a tri-duality form if the primal problem and its canonical dual have the same dimension; otherwise, both the canonical min-max duality and the double-max duality still hold strongly, but the double-min duality holds weakly in a symmetrical form. Four numerical examples are presented to illustrate that this theor…
In recent work arXiv:2109.07820 we have shown the equivalence of the widely used nonconvex (generalized) branched transport problem with a shape optimization problem of a street or railroad network, known as (generalized) urban planning problem. The argument was solely based on an explicit construction and characterization of competitors. In the current article we instead analyse the dual perspective associated with both problems. In more detail, the shape optimization problem involves the Wasserstein distance between two measures with respect to a metric depending on the street network. We sh…
Optimization problems with discrete decisions are nonconvex and thus lack strong duality, which limits the usefulness of tools such as shadow prices and the KKT conditions. It was shown in Burer(2009) that mixed-binary quadratic programs can be written as completely positive programs, which are convex. Completely positive reformulations of discrete optimization problems therefore have strong duality if a constraint qualification is satisfied. We apply this perspective in two ways. First, we write unit commitment in power systems as a completely positive program, and use the dual copositive pro…
