We analyze combinatorial optimization problems with ordinal, i.e., non-additive, objective functions that assign categories (like good, medium and bad) rather than cost coefficients to the elements of feasible solutions. We review different optimality concepts for ordinal optimization problems and discuss their similarities and differences. We then focus on two prevalent optimality concepts that are shown to be equivalent. Our main result is a bijective linear transformation that transforms ordinal optimization problems to associated standard multi-objective optimization problems with binary cost coefficients. Since this transformation preserves all properties of the underlying problem, problem-specific solution methods remain applicable. A prominent example is dynamic programming and Bellman's principle of optimality, that can be applied, e.g., to ordinal shortest path and ordinal knapsack problems. We extend our results to multi-objective optimization problems that combine ordinal and real-valued objective functions.
Bi-objective optimization problems on matroids are in general intractable and their corresponding decision problems are in general NP-hard. However, if one of the objective functions is restricted to binary cost coefficients the problem becomes efficiently solvable by an exhaustive swap algorithm. Binary cost coefficients often represent two categories and are thus a special case of ordinal coefficients that are in general non-additive. In this paper we consider ordinal objective functions with more than two categories in the context of matroid optimization. We introduce several problem varian…
We consider in this paper a class of semi-continuous quadratic programming problems which arises in many real-world applications such as production planning, portfolio selection and subset selection in regression. We propose a lift-and-convexification approach to derive an equivalent reformulation of the original problem. This lift-and-convexification approach lifts the quadratic term involving $x$ only in the original objective function $f(x,y)$ to a quadratic function of both $x$ and $y$ and convexifies this equivalent objective function. While the continuous relaxation of our new reformulat…
We consider a multiobjective bilevel optimization problem with vector-valued upper- and lower-level objective functions. Such problems have attracted a lot of interest in recent years. However, so far, scalarization has appeared to be the main approach used to deal with the lower-level problem. Here, we utilize the concept of frontier map that extends the notion of optimal value function to our parametric multiobjective lower-level problem. Based on this, we build a tractable constraint qualification that we use to derive necessary optimality conditions for the problem. Subsequently, we show t…
Bayesian optimisation is a powerful tool to solve expensive black-box problems, but fails when the stationary assumption made on the objective function is strongly violated, which is the case in particular for ill-conditioned or discontinuous objectives. We tackle this problem by proposing a new Bayesian optimisation framework that only considers the ordering of variables, both in the input and output spaces, to fit a Gaussian process in a latent space. By doing so, our approach is agnostic to the original metrics on the original spaces. We propose two algorithms, respectively based on an opti…
We investigate in this paper the generalized trust region subproblem (GTRS) of minimizing a general quadratic objective function subject to a general quadratic inequality constraint. By applying a simultaneous block diagonalization approach, we obtain a congruent canonical form for the symmetric matrices in both the objective and constraint functions. By exploiting the block separability of the canonical form, we show that all GTRSs with an optimal value bounded from below are second order cone programming (SOCP) representable. Our result generalizes the recent work of Ben-Tal and Hertog (Math…
There is a recent surge of interest in nonconvex reformulations via low-rank factorization for stochastic convex semidefinite optimization problem in the purpose of efficiency and scalability. Compared with the original convex formulations, the nonconvex ones typically involve much fewer variables, allowing them to scale to scenarios with millions of variables. However, it opens a new challenge that under what conditions the nonconvex stochastic algorithms may find the global optima effectively despite their empirical success in applications. In this paper, we provide an answer that a stochast…
In this paper we introduce a new parameterized Quadratic Decision Rule (QDR), a generalisation of the commonly employed Affine Decision Rule (ADR), for two-stage linear adjustable robust optimization problems with ellipsoidal uncertainty and show that (affinely parameterized) linear adjustable robust optimization problems with QDRs are numerically tractable by presenting exact semi-definite program (SDP) and second order cone program (SOCP) reformulations. Under these QDRs, we also establish that exact conic program reformulations also hold for two-stage linear ARO problems, containing also ad…
We present a geometrical analysis on the completely positive programming reformulation of quadratic optimization problems and its extension to polynomial optimization problems with a class of geometrically defined nonconvex conic programs and their covexification. The class of nonconvex conic programs is described with a linear objective functionin a linear space $V$, and the constraint set is represented geometrically as the intersection of a nonconvex cone $K \subset V$, a face $J$ of the convex hull of $K$ and a parallel translation $L$ of a supporting hyperplane of the nonconvex cone $K$. …
We present algorithm MIQCR-CB that is an advancement of method MIQCR~(Billionnet, Elloumi and Lambert, 2012). MIQCR is a method for solving mixed-integer quadratic programs and works in two phases: the first phase determines an equivalent quadratic formulation with a convex objective function by solving a semidefinite problem $(SDP)$, and, in the second phase, the equivalent formulation is solved by a standard solver. As the reformulation relies on the solution of a large-scale semidefinite program, it is not tractable by existing semidefinite solvers, already for medium sized problems. To sur…
We revisit the optimal control problem with maximum cost with the objective to provide different equivalent reformulations suitable to numerical methods. We propose two reformulations in terms of extended Mayer problems with constraint, and another one in terms of a differential inclusion with upper-semi continuous right member but without constraint. For this last one we also propose an approximation scheme of the optimal value from below. These approaches are illustrated and discussed on several examples.
We consider the sparse optimization problem with nonlinear constraints and an objective function, which is given by the sum of a general smooth mapping and an additional term defined by the $ \ell_0 $-quasi-norm. This term is used to obtain sparse solutions, but difficult to handle due to its nonconvexity and nonsmoothness (the sparsity-improving term is even discontinuous). The aim of this paper is to present two reformulations of this program as a smooth nonlinear program with complementarity-type constraints. We show that these programs are equivalent in terms of local and global minima and…
This paper is concerned with objective value performance of the scenario approach for robust convex optimization. A novel method is proposed to derive probabilistic bounds for the objective value from scenario programs with a finite number of samples. This method relies on a max-min reformulation and the concept of complexity of robust optimization problems. With additional continuity and regularity conditions, via sensitivity analysis, we also provide explicit bounds which outperform an existing result in the literature. To illustrate the improvements of our results, we also provide a numeric…
Quantum devices can be used to solve constrained combinatorial optimization (COPT) problems thanks to the use of penalization methods to embed the COPT problem's constraints in its objective to obtain a quadratic unconstrained binary optimization (QUBO) reformulation of the COPT. However, the particular way in which this penalization is carried out, affects the value of the penalty parameters, as well as the number of additional binary variables that are needed to obtain the desired QUBO reformulation. In turn, these factors substantially affect the ability of quantum computers to efficiently …
The preferences of players in non-cooperative games represent their choice in the set of available options, which meet the completeness property if players are able to compare any pair of available options. In the existing literature, the variational reformulation of generalized Nash games relies on the numerical representation of players' preferences into objective functions, which is only possible if preferences are transitive and complete. In this work, we first characterize the jointly convex generalized Nash equilibrium problems in terms of a variational inequality without requiring any n…
A DC-Reformulation for Gradient-$L^0$-Constrained Problems
Würzburg University（维尔茨堡大学）；Interdisciplinary Center for Scientific Computing, Heidelberg University（海德堡大学跨学科科学计算中心）；Institute for Mathematics, Heidelberg University（海德堡大学数学研究所）
Exploring New Frontiers in Vertical Federated Learning: the Role of Saddle Point Reformulation
Basic Research of Artificial Intelligence Laboratory (BRAIn Lab)（人工智能基础研究实验室（BRAIn实验室））；Federated Learning Problems Laboratory（联邦学习问题实验室）；Innopolis University（Innopolis大学）；Sber AI Lab（俄联邦储蓄银行人工智能实验室）；Technical University of Munich（慕尼黑工业大学）；HSE University（高等经济大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Hong Kong University of Science and Technology（香港科技大学）
The problem of estimating sparse eigenvectors of a symmetric matrix attracts a lot of attention in many applications, especially those with high dimensional data set. While classical eigenvectors can be obtained as the solution of a maximization problem, existing approaches formulate this problem by adding a penalty term into the objective function that encourages a sparse solution. However, the resulting methods achieve sparsity at the expense of sacrificing the orthogonality property. In this paper, we develop a new method to estimate dominant sparse eigenvectors without trading off their or…
In this work, we consider a constrained convex problem with linear inequalities and provide an inexact penalty re-formulation of the problem. The novelty is in the choice of the penalty functions, which are smooth and can induce a non-zero penalty over some points in feasible region of the original constrained problem. The resulting unconstrained penalized problem is parametrized by two penalty parameters which control the slope and the curvature of the penalty function. With a suitable selection of these penalty parameters, we show that the solutions of the resulting penalized unconstrained p…
In decentralized optimization, several nodes connected by a network collaboratively minimize some objective function. For minimization of Lipschitz functions lower bounds are known along with optimal algorithms. We study a specific class of problems: linear models with nonsmooth loss functions. Our algorithm combines regularization and dual reformulation to get an effective optimization method with complexity better than the lower bounds.
Tencent AI Lab（腾讯人工智能实验室）；Hong Kong University of Science and Technology（香港科技大学）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）
Maximum a posteriori (MAP) inference is an important task for graphical models. Due to complex dependencies among variables in realistic model, finding an exact solution for MAP inference is often intractable. Thus, many approximation methods have been developed, among which the linear programming (LP) relaxation based methods show promising performance. However, one major drawback of LP relaxation is that it is possible to give fractional solutions. Instead of presenting a tighter relaxation, in this work we propose a continuous but equivalent reformulation of the original MAP inference probl…
For minimizing a strongly convex objective function subject to linear inequality constraints, we consider a penalty approach that allows one to utilize stochastic methods for problems with a large number of constraints and/or objective function terms. We provide upper bounds on the distance between the solutions to the original constrained problem and the penalty reformulations, guaranteeing the convergence of the proposed approach. We give a nested accelerated stochastic gradient method and propose a novel way for updating the smoothness parameter of the penalty function and the step-size. Th…
Security proof methods for quantum key distribution, QKD, that are based on the numerical key rate calculation problem, are powerful in principle. However, the practicality of the methods are limited by computational resources and the efficiency and accuracy of the underlying algorithms for convex optimization. We derive a stable reformulation of the convex nonlinear semidefinite programming, SDP, model for the key rate calculation problems. We use this to develop an efficient, accurate algorithm. The stable reformulation is based on novel forms of facial reduction, FR, for both the linear con…
Nonconvex Nonsmooth Multicomposite Optimization and Its Applications to Recurrent Neural Networks
Department of Applied Mathematics, Hong Kong Polytechnic University（香港理工大学应用数学系）；School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）
Distributionally robust optimization (DRO) is a widely used framework for optimizing objective functionals in the presence of both randomness and model-form uncertainty. A key step in the practical solution of many DRO problems is a tractable reformulation of the optimization over the chosen model ambiguity set, which is generally infinite dimensional. Previous works have solved this problem in the case where the objective functional is an expected value. In this paper we study objective functionals that are the sum of an expected value and a variance penalty term. We prove that the correspond…
In decentralized consensus optimization, a connected network of agents collaboratively minimize the sum of their local objective functions over a common decision variable, where their information exchange is restricted between the neighbors. To this end, one can first obtain a problem reformulation and then apply the alternating direction method of multipliers (ADMM). The method applies iterative computation at the individual agents and information exchange between the neighbors. This approach has been observed to converge quickly and deemed powerful. This paper establishes its linear converge…
