The bilevel program is an optimization problem where the constraint involves solutions to a parametric optimization problem. It is well-known that the value function reformulation provides an equivalent single-level optimization problem but it results in a nonsmooth optimization problem which never satisfies the usual constraint qualification such as the Mangasarian-Fromovitz constraint qualification (MFCQ). In this paper we show that even the first order sufficient condition for metric subregularity (which is in general weaker than MFCQ) fails at each feasible point of the bilevel program. We introduce the concept of directional calmness condition and show that under {the} directional calmness condition, the directional necessary optimality condition holds. {While the directional optimality condition is in general sharper than the non-directional one,} the directional calmness condition is in general weaker than the classical calmness condition and hence is more likely to hold. {We perform the directional sensitivity analysis of the value function and} propose the directional quasi-normality as a sufficient condition for the directional calmness. An example is given to show that the directional quasi-normality condition may hold for the bilevel program.
In this paper we study constraint qualifications and optimality conditions for bilevel programming problems. We strive to derive checkable constraint qualifications in terms of problem data and applicable optimality conditions. For the bilevel program with convex lower level program we discuss drawbacks of reformulating a bilevel programming problem by the mathematical program with complementarity constraints and present a new sharp necessary optimality condition for the reformulation by the mathematical program with a generalized equation constraint. For the bilevel program with a nonconvex l…
Here, necessary optimal condition for Optimistic Bilevel programming problem is obtained in Asplund spaces. Also we have got necessary optimal conditions in finite dimensional spaces, by assuming differentiability on the given functions.
This paper is concerned with the derivation of first- and second-order sufficient optimality conditions for optimistic bilevel optimization problems involving smooth functions. First-order sufficient optimality conditions are obtained by estimating the tangent cone to the feasible set of the bilevel program in terms of initial problem data. This is done by exploiting several different reformulations of the hierarchical model as a single-level problem. To obtain second-order sufficient optimality conditions, we exploit the so-called value function reformulation of the bilevel optimization probl…
In this paper, we propose a combined approach with second-order optimality conditions of the lower level problem to study constraint qualifications and optimality conditions for bilevel programming problems. The new method is inspired by the combined approach developed by Ye and Zhu in 2010, where the authors combined the classical first-order and the value function approaches to derive new necessary optimality conditions. In our approach, we add a second-order optimality condition to the combined program as a new constraint. We show that when all known approaches fail, adding the second-order…
We address the problem of optimal scale-dependent parameter learning in total variation image denoising. Such problems are formulated as bilevel optimization instances with total variation denoising problems as lower-level constraints. For the bilevel problem, we are able to derive M-stationarity conditions, after characterizing the corresponding Mordukhovich generalized normal cone and verifying suitable constraint qualification conditions. We also derive B-stationarity conditions, after investigating the Lipschitz continuity and directional differentiability of the lower-level solution opera…
Directional first order approach for a class of bilevel programs
School of Mathematics, Hunan University（湖南大学数学学院）；School of Artificial Intelligence, Southern University of Science and Technology（南方科技大学人工智能学院）；Department of Mathematics and Statistics, University of Victoria（维多利亚大学数学与统计系）；Department of Mathematics, Southern University of Science and Technology（南方科技大学数学系）
In this paper we begin by discussing the simple bilevel programming problem (SBP) and its extension the simple mathematical programming problem under equilibrium constraints (SMPEC). Here we first define both these problems and study their interrelations. Next we study the various types of necessary and sufficient optimality conditions for the (SMPEC) problems; which occur under various reformulations. The optimality conditions for the (SBP) problem are special cases of the results obtained here when the lower level objective is the gradient of a convex function. Among the various optimality c…
This article studies Gauss-Newton-type methods for over-determined systems to find solutions to bilevel programming problems. To proceed, we use the lower-level value function reformulation of bilevel programs and consider necessary optimality conditions under appropriate assumptions. First under strict complementarity for upper- and lower-level feasibility constraints, we prove the convergence of a Gauss-Newton-type method in computing points satisfying these optimality conditions under additional tractable qualification conditions. Potential approaches to address the shortcomings of the meth…
We consider a multiobjective bilevel optimization problem with vector-valued upper- and lower-level objective functions. Such problems have attracted a lot of interest in recent years. However, so far, scalarization has appeared to be the main approach used to deal with the lower-level problem. Here, we utilize the concept of frontier map that extends the notion of optimal value function to our parametric multiobjective lower-level problem. Based on this, we build a tractable constraint qualification that we use to derive necessary optimality conditions for the problem. Subsequently, we show t…
This chapter presents a self-contained approach of variational analysis and generalized differentiation to deriving necessary optimality in problems of bilevel optimization with Lipschitzian data. We mainly concentrate on optimistic models, although the developed machinery also applies to pessimistic versions. Some open problems are posed and discussed.
The mathematical modeling of numerous real-world applications results in hierarchical optimization problems with two decision makers where at least one of them has to solve an optimal control problem of ordinary or partial differential equations. Such models are referred to as bilevel optimal control problems. Here, we first review some different features of bilevel optimal control including important applications, existence results, solution approaches, and optimality conditions. Afterwards, we focus on a specific problem class where parameters appearing in the objective functional of an opti…
In this paper we propose a bilevel optimization approach for the placement of space and time observations in variational data assimilation problems. Within the framework of supervised learning, we consider a bilevel problem where the lower-level task is the variational reconstruction of the initial condition of a semilinear system, and the upper-level problem solves the optimal placement with help of a sparsity inducing function. Due to the pointwise nature of the observations, an optimality system with regular Borel measures is obtained as necessary optimality condition for the lower-level pr…
The authors' paper in Optimization 63 (2014), 505-533, see Ref. [5], was the first one to provide detailed optimality conditions for pessimistic bilevel optimization. The results there were based on the concept of the two-level optimal value function introduced and analyzed in SIAM J. Optim. 22 (2012), 1309-1343; see Ref. [4], for the case of optimistic bilevel programs. One of the basic assumptions in both of these papers is that the functions involved in the problems are at least continuously differentiable. Motivated by the fact that many real-world applications of optimization involve func…
In this article, we investigate a time-optimal state-constrained bilevel optimal control problem whose lower-level dynamics feature a sweeping control process involving a {\it truncated normal cone}. By bilevel, it is meant that the optimization of the upper level problem is carried out over the solution set of the lower level problem.This problem instance arises in structured crowd motion control problems in a confined space. We establish the corresponding necessary optimality conditions in the Gamkrelidze's form. The analysis relies on the {\it smooth approximation} of the lower level sweepi…
We propose a bilevel optimization strategy for selecting the best hyperparameter value for the nonsmooth $\ell_p$ regularizer with $0<p\le 1$. The concerned bilevel optimization problem has a nonsmooth, possibly nonconvex, $\ell_p$-regularized problem as the lower-level problem. Despite the recent popularity of nonconvex $\ell_p$-regularizer and the usefulness of bilevel optimization for selecting hyperparameters, algorithms for such bilevel problems have not been studied because of the difficulty of $\ell_p$-regularizer. Our contribution is the proposal of the first algorithm equipped with a …
A learning approach to selecting regularization parameters in multi-penalty Tikhonov regularization is investigated. It leads to a bilevel optimization problem, where the lower level problem is a Tikhonov regularized problem parameterized in the regularization parameters. Conditions which ensure the existence of solutions to the bilevel optimization problem of interest are derived, and these conditions are verified for two relevant examples. Difficulties arising from the possible lack of convexity of the lower level problems are discussed. Optimality conditions are given provided that a reason…
We consider bilevel linear problems, where the right-hand side of the lower level problems is stochastic. The leader has to decide in a here-and-now fashion, while the follower has complete information. In this setting, the leader's outcome can be modeled by a random variable, which gives rise to a broad spectrum of models involving coherent or convex risk measures and stochastic dominance constraints. We outline Lipschitzian properties, conditions for existence and optimality, as well as stability results. Moreover, for finite discrete distributions, we discuss the special structure of equiva…
The partial calmness for the bilevel programming problem (BLPP) is an important condition which ensures that a local optimal solution of BLPP is a local optimal solution of a partially penalized problem where the lower level optimality constraint is moved to the objective function and hence a weaker constraint qualification can be applied. In this paper we propose a sufficient condition in the form of a partial error bound condition which guarantees the partial calmness condition. We analyse the partial calmness for the combined program based on the Bouligand (B-) and the Fritz John (FJ) stati…
Partial calmness is a celebrated but restrictive property of bilevel optimization problems whose presence opens a way to the derivation of Karush--Kuhn--Tucker-type necessary optimality conditions in order to characterize local minimizers. In the past, sufficient conditions for the validity of partial calmness have been investigated. In this regard, the presence of a linearly structured lower level problem has turned out to be beneficial. However, the associated literature suffers from inaccurate results. In this note, we clarify some regarding erroneous statements and visualize the underlying…
Bilevel programs are optimization problems where some variables are solutions to optimization problems themselves, and they arise in a variety of control applications, including: control of vehicle traffic networks, inverse reinforcement learning and inverse optimization, and robust control for human-automation systems. This paper develops a duality-based approach to solving bilevel programs where the lower level problem is convex. Our approach is to use partial dualization to construct a new dual function that is differentiable, unlike the Lagrangian dual that is only directionally differenti…
The most common procedure to solve a linear bilevel problem in the PES community is, by far, to transform it into an equivalent single-level problem by replacing the lower level with its KKT optimality conditions. Then, the complementarity conditions are reformulated using additional binary variables and large enough constants (big-Ms) to cast the single-level problem as a mixed-integer linear program that can be solved using optimization software. In most cases, such large constants are tuned by trial and error. We show, through a counterexample, that this widely used trial-and-error approach…
We consider a bilevel optimization approach in function space for the choice of spatially dependent regularization parameters in TV image restoration models. First- and second-order optimality conditions for the bilevel problem are studied, when the spatially-dependent parameter belongs to the Sobolev space $H^1(Ω)$. A combined Schwarz domain decomposition-semismooth Newton method is proposed for the solution of the full optimality system and local superlinear convergence of the semismooth Newton method is analyzed. Exhaustive numerical computations are finally carried out to show the suitabil…
We introduce and study a notion of directional Pareto minimality with respect to a set that generalizes the classical concept of Pareto efficiency. Then we give separate necessary and sufficient conditions for the newly introduced efficiency and several situations concerning the objective mapping and the constraints are considered. In order to investigate different cases, we adapt some well-known constructions of generalized differentiation and the connections with some recent directional regularities come naturally into play. As a consequence, several techniques from the study of genuine Pare…
We propose a bilevel optimization approach for the estimation of parameters in nonlocal image denoising models. The parameters we consider are both the fidelity weight and weights within the kernel of the nonlocal operator. In both cases we investigate the differentiability of the solution operator in function spaces and derive a first order optimality system that characterizes local minima. For the numerical solution of the problems, we use a second-order trust-region algorithm in combination with a finite element discretization of the nonlocal denoising models and we introduce a computationa…
This article concerns an optimal crowd motion control problem in which the crowd features a structure given by its organization into N groups (participants) each one spatially confined in a set. The overall optimal control problem consists in driving the ensemble of sets as close as possible to a given point (the 'exit') while the population in each set minimizes its control effort subject to its sweeping dynamics with a controlled state dependent velocity drift. In order to capture the conflict between the goal of the overall population and those of the various groups, the problem is cast as …
Relaxed constant positive linear dependence constraint qualification (RCPLD) for a system of smooth equalities and inequalities is a constraint qualification that is weaker than the usual constraint qualifications such as Mangasarian Fromovitz constraint qualification and the linear constraint qualification. Moreover RCPLD is known to induce an error bound property. In this paper we extend RCPLD to a very general feasibility system which may include Lipschitz continuous inequality constraints, complementarity constraints and abstract constraints. We show that this RCPLD for the general system …
Bilevel programming can be used to formulate many problems in the field of power systems, such as strategic bidding. However, common reformulations of bilevel problems to mixed-integer linear programs make solving such problems hard, which impedes their implementation in real-life. In this paper, we significantly improve solution speed and tractability by introducing decision trees to learn the active constraints of the lower-level problem, while avoiding to introduce binaries and big-M constants. The application of machine learning reduces the online solving time, by moving the selection of a…
Generalized Differentiability and Second-Order Necessary Optimality Conditions for an Elliptic Optimal Control Problem with Exponential Nonlinearity and Discrete Measures
