We consider a stochastic control problem where the set of strict (classical) controls is not necessarily convex, and the system is governed by a nonlinear backward stochastic differential equation. By introducing a new approach, we establish necessary as well as sufficient conditions of optimality for two models. The first concerns the relaxed controls, who are measure-valued processes. The second is a particular case of the first and relates to strict control problems.
We consider optimal control problems for systems governed by mean-field stochastic differential equations, where the control enters both the drift and the diffusion coefficient. We study the relaxed model, in which admissible controls are measure-valued processes and the relaxed state process is driven by an orthogonal martingale measure, whose covariance measure is the relaxed control. This is a natural extension of the original strict control problem, for which we prove the existence of an optimal control. Then, we derive optimality necessary conditions for this problem, in terms of two adjo…
We discuss the (first- and second-order) optimality conditions for nonlinear programming under the relaxed constant rank constraint qualification. This condition generalizes the so-called linear independence constraint qualification. Although the optimality conditions are well established in the literature, the proofs presented here are based solely on the well-known inverse function theorem. This is the only prerequisite from real analysis used to establish two auxiliary results needed to prove the optimality conditions, thereby making this paper totally self-contained.
We consider a stochastic control problem where the set of controls is not necessarily convex and the system is governed by a nonlinear backward stochastic differential equation. We establish necessary as well as sufficient conditions of optimality for two models. The first concerns the strict (classical) controls. The second is an extension of the first to relaxed controls, who are a measure valued processes.
We associate with each convex optimization problem posed on some locally convex space with an infinite index set T, and a given non-empty family H formed by finite subsets of T, a suitable Lagrangian-Haar dual problem. We provide reverse H-strong duality theorems, H-Farkas type lemmas and optimality theorems. Special attention is addressed to infinite and semi-infinite linear optimization problems.
This paper is concerned with optimal control problems for systems governed by mean-field stochastic differential equation, in which the control enters both the drift and the diffusion coefficient. We prove that the relaxed state process, associated with measure valued controls, is governed by an orthogonal martingale measure rather that a Brownian motion. In particular, we show by a counter example that replacing the drift and diffusion coefficient by their relaxed counterparts does not define a true relaxed control problem. We establish the existence of an optimal relaxed control, which can b…
We consider in this paper, mixed relaxed-singular stochastic control problems, where the control variable has two components, the first being measure-valued and the second singular. The control domain is not necessarily convex and the system is governed by a nonlinear stochastic differential equation, in which the measure-valued part of the control enters both the drift and the diffusion coefficients. We establish necessary optimality conditions, of the Pontryagin maximum principle type, satisfied by an optimal relaxed-singular control, which exist under general conditions on the coefficients.…
In this paper we consider non convex control problems of stochastic differential equations driven by relaxed controls. We present existence of optimal controls and then develop necessary conditions of optimality. We cover both continuous diffusion and Jump processes.
Mean field games are studied in the framework of controlled martingale problems, and general existence theorems are proven in which the equilibrium control is Markovian. The framework is flexible enough to include degenerate volatility, which may depend on both the control and the mean field. The objectives need not be strictly convex, and the mean field interactions considered are nonlocal and Wasserstein-continuous. When the volatility is nondegenerate, continuity assumptions may be weakened considerably. The proofs first use relaxed controls to establish existence. Then, using a convexity a…
This paper is concerned with optimal control of systems driven by G-stochastic differential equations (G-SDEs), with controlled jump term. We study the relaxed problem, in which admissible controls are measurevalued processes and the state variable is governed by an G-SDE driven by a counting measure valued process called relaxed Poisson measure such that the compensator is a product measure. Under some conditions on the coefficients, using the G-chattering lemma, we show that the strict and the relaxed control problems have the same value function. Additionally, we derive a maximum principle …
We revisit a stochastic control problem of optimally modifying the underlying spectrally negative Levy process. A strategy must be absolutely continuous with respect to the Lebesgue measure, and the objective is to minimize the total costs of the running and controlling costs. Under the assumption that the running cost function is convex, we show the optimality of a refraction strategy. We also obtain the convergence of the optimal refraction strategies and the value functions, as the control set is enlarged, to those in the relaxed case without the absolutely continuous assumption. Numerical …
We study relaxed stochastic control problems where the state equation is a one dimensional linear stochastic differential equation with random and unbounded coefficients. The two main results are existence of an optimal relaxed control and necessary conditions for optimality in the form of a relaxed maximum principle. The main motivation is an optimal bond portfolio problem in a market where there exists a continuum of bonds and the portfolio weights are modeled as measure-valued processes on the set of times to maturity.
We define w-invex set, w-preinvex, w-strictly preinvex, w-quasi preinvex, w-strictly quasi preinvex, w-semi-strictly quasi preinvex, and w-pre pseudo-invex functions in this context. And these form a class of real functions, which is the generalization of a family of preinvex functions. Here, we provide a thorough analysis of the core characteristics of these functions, along with numerous examples that help to illustrate the idea. Finally, w-quasi preinvex, w-strictly preinvex, and w-strictly quasi preinvex functions are used to analyze the optimization problems.
In this paper we study constraint qualifications and optimality conditions for bilevel programming problems. We strive to derive checkable constraint qualifications in terms of problem data and applicable optimality conditions. For the bilevel program with convex lower level program we discuss drawbacks of reformulating a bilevel programming problem by the mathematical program with complementarity constraints and present a new sharp necessary optimality condition for the reformulation by the mathematical program with a generalized equation constraint. For the bilevel program with a nonconvex l…
We consider a control problem where the system is driven by a decoupled as well as a coupled forward-backward stochastic differential equation. We prove the existence of an optimal control in the class of relaxed controls, which are measure-valued processes, generalizing the usual strict controls. The proof is based on some tightness properties and weak convergence on the space D of càdlàg functions, endowed with the Jakubowsky S-topology. Moreover, under some convexity assumptions, we show that the relaxed optimal control is realized by a strict control.
We consider a stochastic control problem where the set of strict (classical) controls is not necessarily convex and the the variable control has two components, the first being absolutely continuous and the second singular. The system is governed by a nonlinear stochastic differential equation, in which the absolutely continuous component of the control enters both the drift and the diffusion coefficients. By introducing a new approach, we establish necessary and sufficient optimality conditions for two models. The first concerns the relaxed-singular controls, who are a pair of processes whose…
In this paper, we establish some second order necessary/sufficient optimality conditions for optimal control problems of stochastic evolution equations in infinite dimensions. The control acts on both the drift and diffusion terms and the control region is convex. The concepts of relaxed and $V$-transposition solutions (introduced in our previous works) to operator-valued backward stochastic evolution equations are employed to derive these optimality conditions. The correction part of the second order adjoint equation, which does not appear in the (first order) Pontryagin-type stochastic maxim…
We study singular stochastic control of a two dimensional stochastic differential equation, where the first component is linear with random and unbounded coefficients. We derive existence of an optimal relaxed control and necessary conditions for optimality in the form of a mixed relaxed-singular maximum principle in a global form. A motivating example is given in the form of an optimal investment and consumption problem with transaction costs, where we consider a portfolio with a continuum of bonds and where the portfolio weights are modeled as measure-valued processes on the set of times to …
This paper proves the existence of optimal stopping times via elementary functional analytic arguments. The problem is first relaxed into a convex optimization problem over a closed convex subset of the unit ball of the dual of a Banach space. The existence of optimal solutions then follows from the Banach--Alaoglu compactness theorem and the Krein--Millman theorem on extreme points of convex sets. This approach seems to give the most general existence results known to date. Applying convex duality to the relaxed problem gives a dual problem and optimality conditions in terms of martingales th…
The purpose of this paper is to establish first and second order necessary optimality conditions for optimal control problems of stochastic evolution equations with control and state constraints. The control acts both in the drift and diffusion terms and the control region is a nonempty closed subset of a separable Hilbert space. We employ some classical set-valued analysis tools and theories of the transposition solution of vector-valued backward stochastic evolution equations and the relaxed-transposition solution of operator-valued backward stochastic evolution equations to derive these opt…
We develop a framework for the analysis of deep neural networks and neural ODE models that are trained with stochastic gradient algorithms. We do that by identifying the connections between control theory, deep learning and theory of statistical sampling. We derive Pontryagin's optimality principle and study the corresponding gradient flow in the form of Mean-Field Langevin dynamics (MFLD) for solving relaxed data-driven control problems. Subsequently, we study uniform-in-time propagation of chaos of time-discretised MFLD. We derive explicit convergence rate in terms of the learning rate, the …
Extended mean-field control problems with Poissonian common noise: Stochastic maximum principle and Hamiltonian-Jacobi-Bellman equation
Xidian University（西安电子科技大学）；University of Science and Technology of China（中国科学技术大学）；Harbin Institute of Technology（哈尔滨工业大学）；The Hong Kong Polytechnic University（香港理工大学）
In this paper, we study the optimal control system driven by stochastic differential equations (SDEs) of mean-field type, in which the control variable has two components, the first being absolutely continuous and the second singular. On the other hand, the coefficients depend on the state of the solution process as well as of its expected value. Moreover, the cost functional is also of mean field type. This makes the control problem time inconsistent in the sense that the Bellman optimality principle does not hold. Our aim is to derive a stochastic maximum principle of optimal control of Pont…
Second-order necessary optimality conditions for nonlinear conic programming problems that depend on a single Lagrange multiplier are usually built under nondegeneracy and strict complementarity. In this paper we establish a condition of such type for two classes of nonlinear conic problems, namely semidefinite and second-order cone programming, assuming Robinson's constraint qualification and a weak constant rank-type property which are, together, strictly weaker than nondegeneracy. Our approach is done via a penalty-based strategy, which is aimed at providing strong global convergence result…
Relaxed constant positive linear dependence constraint qualification (RCPLD) for a system of smooth equalities and inequalities is a constraint qualification that is weaker than the usual constraint qualifications such as Mangasarian Fromovitz constraint qualification and the linear constraint qualification. Moreover RCPLD is known to induce an error bound property. In this paper we extend RCPLD to a very general feasibility system which may include Lipschitz continuous inequality constraints, complementarity constraints and abstract constraints. We show that this RCPLD for the general system …
This paper exemplifies that saturation is an indispensable structure on measure spaces to obtain the existence and characterization of solutions to nonconvex variational problems with integral constraints in Banach spaces and their dual spaces. We provide a characterization of optimality via the maximum principle for the Hamiltonian and an existence result without the purification of relaxed controls, in which the Lyapunov convexity theorem in infinite dimensions under the saturation hypothesis on the underlying measure space plays a crucial role. We also demonstrate that the existence of solu…
A new relaxed variant of interior point method for low-rank semidefinite programming problems is proposed in this paper. The method is a step outside of the usual interior point framework. In anticipation to converging to a low-rank primal solution, a special nearly low-rank form of all primal iterates is imposed. To accommodate such a (restrictive) structure, the first order optimality conditions have to be relaxed and are therefore approximated by solving an auxiliary least-squares problem. The relaxed interior point framework opens numerous possibilities how primal and dual approximated New…
Following Arnold's geometric interpretation, the Euler equations of an incompressible fluid moving in a domain D are known to be the optimality equation of the minimizing geodesic problem along the group of orientation and volume preserving diffeomorphisms of D. This problem admits a well-established convex relaxation which generates a set of "relaxed", "multi-stream", version of the Euler equations. However, it is unclear that such relaxed equations are appropriate for the initial value problem and the theory of turbulence, due to their lack of well-posedness for most initial data. As an atte…
In this paper, in terms of three types of generalized second-order derivatives of a nonsmooth function, we mainly study the corresponding second-order optimality conditions in a Hilbert space and prove the equivalence among these optimality conditions for paraconcave functions. As applications, we use these second-order optimality conditions to study strict local minimizers of order two and provide sufficient and/or necessary conditions for ensuring the local minimizer. This work extends and generalizes the study on second-order optimality conditions from the finite-dimensional space to the Hi…
In the paper we consider the infinite horizon control problems on the interval with free right-hand endpoint. We obtain the necessary conditions of strict optimality. The method of the proof actually follows the classic paper by Halkin, and the boundary condition for infinity that we construct in our paper is a stronger variety of the Seierstad condition. The complete system of relations of the maximum principle that was obtained in the paper allows us to write the expression for the adjoint variable in the form of improper integral that depends only on the developing trajectory. S.M. Aseev, A…
In this paper, we introduce a kind of approximate Karush--Kuhn--Tucker condition (AKKT) for a smooth cone-constrained vector optimization problem. We show that, without any constraint qualification, the AKKT condition is a necessary for a local weak efficient solution of the considered problem. For convex problems, we prove that the AKKT condition is a necessary and sufficient optimality condition for a global weak efficient solution. We also introduce some strict constraint qualifications associated with the AKKT condition.
We consider average-cost Markov decision processes (MDPs) with Borel state spaces, countable, discrete action spaces, and strictly unbounded one-stage costs. For the minimum pair approach, we introduce a new majorization condition on the state transition stochastic kernel, in place of the commonly required continuity conditions on the MDP model. We combine this majorization condition with Lusin's theorem to prove the existence of a stationary minimum pair, i.e., a stationary policy paired with an invariant probability measure induced on the state space, with the property that the pair attains …
Stochastic maximum principle of nonlinear controlled forward-backward systems, where the set of strict (classical) controls need not be convex and the diffusion coefficient depends explicitly on the variable control, is an open problem impossible to solve by the classical method of spike variation. In this paper, we introduce a new approach to solve this open problem and we establish necessary as well as sufficient conditions of optimality, in the form of global stochastic maximum principle, for two models. The first concerns the relaxed controls, who are a measure-valued processes. The second…
The cardinality constrained optimization problem (CCOP) is an optimization problem where the maximum number of nonzero components of any feasible point is bounded. In this paper, we consider CCOP as a mathematical program with disjunctive subspaces constraints (MPDSC). Since a subspace is a special case of a convex polyhedral set, MPDSC is a special case of the mathematical program with disjunctive constraints (MPDC). Using the special structure of subspaces, we are able to obtain more precise formulas for the tangent and (directional) normal cones for the disjunctive set of subspaces. We then…
University of Heidelberg（海德堡大学）；University of Cambridge（剑桥大学）
We consider a variational convex relaxation of a class of optimal partitioning and multiclass labeling problems, which has recently proven quite successful and can be seen as a continuous analogue of Linear Programming (LP) relaxation methods for finite-dimensional problems. While for the latter case several optimality bounds are known, to our knowledge no such bounds exist in the continuous setting. We provide such a bound by analyzing a probabilistic rounding method, showing that it is possible to obtain an integral solution of the original partitioning problem from a solution of the relaxed…
We consider nonlinear model predictive control (MPC) with multiple competing cost functions. In each step of the scheme, a multiobjective optimal control problem with a nonlinear system and terminal conditions is solved. We propose an algorithm and give performance guarantees for the resulting MPC closed loop system. Thereby, we significantly simplify the assumptions made in the literature so far by assuming strict dissipativity and the existence of a compatible terminal cost for one of the competing objective functions only. We give conditions which ensure asymptotic stability of the closed l…
Lasserre's hierarchy is a sequence of semidefinite relaxations for solving polynomial optimization problems globally. This paper studies the relationship between optimality conditions in nonlinear programming theory and finite convergence of Lasserre's hierarchy. Our main results are: i) Lasserre's hierarchy has finite convergence when the constraint qualification, strict complementarity and second order sufficiency conditions hold at every global minimizer, under the standard archimedean assumption; the proof uses a result of Marshall on boundary hessian conditions. ii) these optimality condi…
Maximization and minimization problems of the principle eigenvalue for divergence form second order elliptic operators with the Dirichlet boundary condition are considered. The principal eigen map of such elliptic operators is introduced and some basic properties of this map, including continuity, concavity, and differentiability with respect to the parameter in the diffusibility matrix, are established. For maximization problem, the admissible control set is convexified to get the existence of an optimal convexified relaxed solution. Whereas, for minimization problem, the relaxation of the pr…
When the objective function is not locally Lipschitz, constraint qualifications are no longer sufficient for Karush-Kuhn-Tucker (KKT) conditions to hold at a local minimizer, let alone ensuring an exact penalization. In this paper, we extend quasi-normality and relaxed constant positive linear dependence (RCPLD) condition to allow the non-Lipschitzness of the objective function and show that they are sufficient for KKT conditions to be necessary for optimality. Moreover, we derive exact penalization results for the following two special cases. When the non-Lipschitz term in the objective funct…
This paper deals with second-order optimality conditions for a quasilinear elliptic control problem with a nonlinear coefficient in the principal part that is countably $PC^2$ (continuous and $C^2$ apart from countably many points). We prove that the control-to-state operator is continuously differentiable even though the nonlinear coefficient is non-smooth. This enables us to establish "no-gap" second-order necessary and sufficient optimality conditions in terms of an abstract curvature functional, i. e., for which the sufficient condition only differs from the necessary one in the fact that …
This paper deals with Pareto solutions of a nonsmooth fractional interval-valued multiobjective optimization. We first introduce four types of Pareto solutions of the considered problem by considering the lower-upper interval order relation and then apply some advanced tools of variational analysis and generalized differentiation to establish necessary optimality conditions for these solutions. Sufficient conditions for Pareto solutions of such a problem are also provided by means of introducing the concepts of (strictly) generalized convex functions defined in terms of the limiting/Mordukhovi…
The design of minimum-compliance bending-resistant structures with continuous cross-section parameters is a challenging task because of its inherent non-convexity. Our contribution develops a strategy that facilitates computing all guaranteed globally optimal solutions for frame and shell structures under multiple load cases and self-weight. To this purpose, we exploit the fact that the stiffness matrix is usually a polynomial function of design variables, allowing us to build an equivalent non-linear semidefinite programming formulation over a semi-algebraic feasible set. This formulation is …
We derive necessary optimality conditions for minimizers of regular functionals in the calculus of variations under smooth state constraints. In the literature, this classical problem is widely investigated. The novelty of our result lies in the fact that the presence of state constraints enters the Euler-Lagrange equations as a local feedback, which allows to derive the $C^{1,1}$-smoothness of solutions. As an application, we discuss a constrained Mean Field Games problem, for which our optimality conditions allow to construct Lipschitz relaxed solutions, thus improving the existence result i…
This paper proposes a joint decomposition method that combines La- grangian decomposition and generalized Benders decomposition, to efficiently solve multiscenario nonconvex mixed-integer nonlinear programming (MINLP) problems to global optimality, without the need for explicit branch and bound search. In this approach, we view the variables coupling the scenario dependent variables and those causing nonconvexity as complicating variables. We systemat- ically solve the Lagrangian decomposition subproblems and the generalized Ben- ders decomposition subproblems in a unified framework. The metho…
Optimal control remains as one of the most versatile frameworks in systems theory, enabling applications ranging from classical robust control to real-time safe operation of fleets of vehicles. While some optimal control problems can be efficiently solved using algebraic or convex methods, most general forms of optimal control must be solved using memory- expensive numerical methods. In this paper we present a theoretical formulation and a corresponding numerical algorithm that can find Pontryagin- optimal inputs for general dynamical systems by using a direct method. Pontryagin-optimal inputs…
Decentralized optimization of distributed stochastic differential systems has been an active area of research for over half a century. Its formulation utilizing static team and person-by-person optimality criteria is well investigated. However, the results have not been generalized to nonlinear distributed stochastic differential systems possibly due to technical difficulties inherent with decentralized decision strategies. In this first part of the two-part paper, we derive team optimality and person-by-person optimality conditions for distributed stochastic differential systems with differen…
We study convergence properties of Stochastic Gradient Descent (SGD) for convex objectives without assumptions on smoothness or strict convexity. We consider the question of establishing that with high probability the objective evaluated at the candidate minimizer returned by SGD is close to the minimal value of the objective. We compare this result concerning the final candidate minimzer (i.e. the final model parameters learned after all gradient steps) to the online learning techniques of [Zin03] that take a rolling average of the model parameters at the different steps of SGD.
We introduce the problem of parameter identification for a coupled nonlocal Cahn-Hilliard-reaction-diffusion PDE system stemming from a recently introduced tumor growth model. The inverse problem of identifying relevant parameters is studied here by relying on techniques from optimal control theory of PDE systems. The parameters to be identified play the role of controls, and a suitable cost functional of tracking-type is introduced to account for the discrepancy between some a priori knowledge of the parameters and the controls themselves. The analysis is carried out for several classes of mo…
We characterize generalized derivatives of the solution operator of the obstacle problem. This precise characterization requires the usage of the theory of so-called capacitary measures and the associated solution operators of relaxed Dirichlet problems. The generalized derivatives can be used to obtain a novel necessary optimality condition for the optimal control of the obstacle problem with control constraints. A comparison shows that this system is stronger than the known system of C-stationarity.
For the model of constrained multi-armed bandit, we show that by construction there exists an index-based deterministic asymptotically optimal algorithm. The optimality is achieved by the convergence of the probability of choosing an optimal feasible arm to one over infinite horizon. The algorithm is built upon Locatelli et al.'s "anytime parameter-free thresholding" algorithm under the assumption that the optimal value is known. We provide a finite-time bound to the probability of the asymptotic optimality given as 1-O(|A|Te^{-T}) where T is the horizon size and A is the set of the arms in th…
We propose a necessary and sufficient test to determine whether a solution for a general quadratic program with two quadratic constraints (QC2QP) can be computed from that of a specific convex semidefinite relaxation, in which case we say that there is no optimality gap. Originally intended to solve a nonconvex optimal control problem, we consider the case in which the cost and both constraints of the QC2QP may be nonconvex. We obtained our test, which also ascertains when strong duality holds, by generalizing a closely-related method by Ai and Zhang. An extension was necessary because, while …
