This paper is devoted to the analysis of a finite horizon discrete-time stochastic optimal control problem, in presence of constraints. We study the regularity of the value function which comes from the dynamic programming algorithm. We derive accurate estimates of the Lipschitz constant of the value function, by means of a regularity result of the multifunction that defines the admissible control set. In the last section we discuss an application to an optimal asset-allocation problem.
This paper studies high-order evaluation complexity for partially separable convexly-constrained optimization involving non-Lipschitzian group sparsity terms in a nonconvex objective function. We propose a partially separable adaptive regularization algorithm using a $p$-th order Taylor model and show that the algorithm can produce an (epsilon,delta)-approximate q-th-order stationary point in at most O(epsilon^{-(p+1)/(p-q+1)}) evaluations of the objective function and its first p derivatives (whenever they exist). Our model uses the underlying rotational symmetry of the Euclidean norm functio…
Asymptotic stationarity and regularity conditions turned out to be quite useful to study the qualitative properties of numerical solution methods for standard nonlinear and complementarity-constrained programs. In this paper, we first extend these notions to nonlinear optimization problems with nonsmooth but Lipschitzian data functions in order to find reasonable notions of asymptotic stationarity and regularity in terms of Clarke's and Mordukhovich's subdifferential construction. Particularly, we compare the associated novel asymptotic constraint qualifications with already existing ones. The…
An adaptive regularization algorithm using high-order models is proposed for partially separable convexly constrained nonlinear optimization problems whose objective function contains non-Lipschitzian $\ell_q$-norm regularization terms for $q\in (0,1)$. It is shown that the algorithm using an $p$-th order Taylor model for $p$ odd needs in general at most $O(ε^{-(p+1)/p})$ evaluations of the objective function and its derivatives (at points where they are defined) to produce an $ε$-approximate first-order critical point. This result is obtained either with Taylor models at the price of requirin…
This paper is concerned with the rank constrained optimization problem whose feasible set is the intersection of the rank constraint set $\mathcal{R}=\!\big\{X\in\mathbb{X}\ |\ {\rm rank}(X)\le κ\big\}$ and a closed convex set $Ω$. We establish the local (global) Lipschitzian type error bounds for estimating the distance from any $X\in Ω$ ($X\in\mathbb{X}$) to the feasible set and the solution set, respectively, under the calmness of a multifunction associated to the feasible set at the origin, which is specially satisfied by three classes of common rank constrained optimization problems. As a…
In these notes, we present a general result concerning the Lipschitz regularity of a certain type of set-valued maps often found in constrained optimization and control problems. The class of multifunctions examined in this paper is characterized by means of a set of Lipschitz continuous constraint functions defined on some Lipschitz manifold. The proof of the regularity result for this class of multifunctions is based on a quantitative version of the Implicit Function Theorem for Lipschitzian maps which provides estimates for the neighborhoods where the implicit map can be defined.
Minimax optimization problems are an important class of optimization problems arising from both modern machine learning and from traditional research areas. We focus on the stability of constrained minimax optimization problems based on the notion of local minimax point by Dai and Zhang (2020). Firstly, we extend the classical Jacobian uniqueness conditions of nonlinear programming to the constrained minimax problem and prove that this set of properties is stable with respect to small $C^2$ perturbation. Secondly, we provide a set of conditions, called Property A, which does not require the st…
This paper introduces the Fejér-monotone hybrid steepest descent method (FM-HSDM), a new member to the HSDM family of algorithms, for solving affinely constrained minimization tasks in real Hilbert spaces, where convex smooth and non-smooth losses compose the objective function. FM-HSDM offers sequences of estimates which converge weakly and, under certain hypotheses, strongly to solutions of the task at hand. Fixed-point theory, variational inequalities and affine-nonexpansive mappings are utilized to devise a scheme that accommodates affine constraints in a more versatile way than state-of-t…
We consider bilevel linear problems, where the right-hand side of the lower level problems is stochastic. The leader has to decide in a here-and-now fashion, while the follower has complete information. In this setting, the leader's outcome can be modeled by a random variable, which gives rise to a broad spectrum of models involving coherent or convex risk measures and stochastic dominance constraints. We outline Lipschitzian properties, conditions for existence and optimality, as well as stability results. Moreover, for finite discrete distributions, we discuss the special structure of equiva…
In this article, we investigate a time-optimal state-constrained bilevel optimal control problem whose lower-level dynamics feature a sweeping control process involving a {\it truncated normal cone}. By bilevel, it is meant that the optimization of the upper level problem is carried out over the solution set of the lower level problem.This problem instance arises in structured crowd motion control problems in a confined space. We establish the corresponding necessary optimality conditions in the Gamkrelidze's form. The analysis relies on the {\it smooth approximation} of the lower level sweepi…
In this article, we generalize the theory of discrete Lagrangian mechanics and variational integrators in two principal directions. First, we show that Lagrangian submanifolds of symplectic groupoids give rise to discrete dynamical systems, and we study the properties of these systems, including their regularity and reversibility, from the perspective of symplectic and Poisson geometry. Next, we use this framework -- along with a generalized notion of generating function due to Sniatycki and Tulczyjew -- to develop a theory of discrete constrained Lagrangian mechanics. This allows for systems …
The main goal of this paper is to apply the machinery of variational analysis and generalized differentiation to study infinite horizon stochastic dynamic programming (DP) with discrete time in the Banach space setting without convexity assumptions. Unlike to standard stochastic DP with stationary Markov processes, we investigate here stochastic DP in $L^p$ spaces to deal with nonstationary stochastic processes, which describe a more flexible learning procedure for the decision-maker. Our main concern is to calculate generalized subgradients of the corresponding value function and to derive ne…
The paper addresses a new class of optimal control problems governed by the dissipative and discontinuous differential inclusion of the sweeping/Moreau process while using controls to determine the best shape of moving convex polyhedra in order to optimize the given Bolza-type functional, which depends on control and state variables as well as their velocities. Besides the highly non-Lipschitzian nature of the unbounded differential inclusion of the controlled sweeping process, the optimal control problems under consideration contain intrinsic state constraints of the inequality and equality t…
This paper proposes a new state estimator for discrete-time nonlinear dynamical systems with unknown-but-bounded uncertainties and state linear inequality and nonlinear equality constraints. Our algorithm is based on constrained zonotopes (CZs) and on a DC programming approach (DC stands for difference of convex functions). Recently, mean value extension and first-order Taylor extension have been adapted from zonotopes to propagate CZs over nonlinear mappings. Although the resulting algorithms (called CZMV and CZFO) reach better precision than the original zonotopic versions, they carry the se…
Discrete black-box optimization problems are challenging for model-based optimization (MBO) algorithms, such as Bayesian optimization, due to the size of the search space and the need to satisfy combinatorial constraints. In particular, these methods require repeatedly solving a complex discrete global optimization problem in the inner loop, where popular heuristic inner-loop solvers introduce approximations and are difficult to adapt to combinatorial constraints. In response, we propose NN+MILP, a general discrete MBO framework using piecewise-linear neural networks as surrogate models and mi…
In this paper, a distributed velocity-constrained consensus problem is studied for discrete-time multi-agent systems, where each agent's velocity is constrained to lie in a nonconvex set. A distributed constrained control algorithm is proposed to enable all agents to converge to a common point using only local information. {The gains of the algorithm for all agents need not to be the same or predesigned and can be adjusted by each agent itself based on its own and neighbors' information.} It is shown that the algorithm is robust to arbitrarily bounded communication delays and arbitrarily switc…
In this paper, we consider a class of optimal control problems for a one-dimensional time-discrete constrained quasilinear diffusion state-systems of singular Allen--Cahn types and its regularized approximating problems. We note that the control parameter for each system is given by physical temperature. The principal part of this paper is started with the verification of a Key-Theorem dealing with the decompositions of the subdifferentials of the governing convex energies of the state-systems. On this basis, we will prove five Main Theorems, concerned with: the solvability and precise regular…
We consider the problem of zeroing an error output of a nonlinear discrete-time system in the presence of constant exogenous disturbances, subject to hard convex constraints on the input signal. The design specification is formulated as a variational inequality, and we adapt a forward-backward splitting algorithm to act as an integral controller which ensures that the input constraints are met at each time step. We establish a low-gain stability result for the closed-loop system when the plant is exponentially stable, generalizing previously known results for integral control of discrete-time …
Constrained discrete optimization problems are encountered in many areas of communication and machine learning. We consider the case where the objective function satisfies Bellman's optimality principle without the constraints on which we place no conditions. We first show that these problems are a generalization of optimization in constrained Markov decision processes with finite horizon used in reinforcement learning and are NP-Hard. We then present a novel multi-survivor dynamic programming (msDP) algorithm that guarantees optimality at significant computational savings. We demonstrate this…
Though switched dynamical systems have shown great utility in modeling a variety of physical phenomena, the construction of an optimal control of such systems has proven difficult since it demands some type of optimal mode scheduling. In this paper, we devise an algorithm for the computation of an optimal control of constrained nonlinear switched dynamical systems. The control parameter for such systems include a continuous-valued input and discrete-valued input, where the latter corresponds to the mode of the switched system that is active at a particular instance in time. Our approach, which…
This papers deals with the constrained discounted control of piecewise deterministic Markov process (PDMPs) in general Borel spaces. The control variable acts on the jump rate and transition measure, and the goal is to minimize the total expected discounted cost, composed of positive running and boundary costs, while satisfying some constraints also in this form. The basic idea is, by using the special features of the PDMPs, to re-write the problem via an embedded discrete-time Markov chain associated to the PDMP and re-formulate the problem as an infinite dimensional linear programming (LP) p…
This work proposes an accelerated primal-dual dynamical system for affine constrained convex optimization and presents a class of primal-dual methods with nonergodic convergence rates. In continuous level, exponential decay of a novel Lyapunov function is established and in discrete level, implicit, semi-implicit and explicit numerical discretizations for the continuous model are considered sequentially and lead to new accelerated primal-dual methods for solving linearly constrained optimization problems. Special structures of the subproblems in those schemes are utilized to develop efficient …
In this paper, we consider the problem of computing parameters of an objective function for a discrete-time optimal control problem from state and control trajectories with active control constraints. We propose a novel method of inverse optimal control that has a computationally efficient online form in which pairs of states and controls from given state and control trajectories are processed sequentially without being stored or processed in batches. We establish conditions guaranteeing the uniqueness of the objective-function parameters computed by our proposed method from trajectories with …
Due to the increasing demand for high performance and cost reduction within the framework of complex system design, numerical optimization of computationally costly problems is an increasingly popular topic in most engineering fields. In this paper, several variants of the Efficient Global Optimization algorithm for costly constrained problems depending simultaneously on continuous decision variables as well as on quantitative and/or qualitative discrete design parameters are proposed. The adaptation that is considered is based on a redefinition of the Gaussian Process kernel as a product betw…
In this paper, we present a new control policy parametrization for the finite-horizon covariance steering problem for discrete-time Gaussian linear systems (DTGLS) which can reduce the latter stochastic optimal control problem to a tractable optimization problem. The covariance steering problem seeks for a feedback control policy that will steer the state covariance of a DTGLS to a desired positive definite matrix in finite time. We consider two different formulations of the covariance steering problem, one with hard terminal LMI constraints (hard-constrained covariance steering) and another o…
The discrete-time mean-variance portfolio selection formulation, a representative of general dynamic mean-risk portfolio selection problems, does not satisfy time consistency in efficiency (TCIE) in general, i.e., a truncated pre-committed efficient policy may become inefficient when considering the corresponding truncated problem, thus stimulating investors' irrational investment behavior. We investigate analytically effects of portfolio constraints on time consistency of efficiency for convex cone constrained markets. More specifically, we derive the semi-analytical expressions for the pre-c…
The main goal of this paper is developing the method of discrete approximations to derive necessary optimality conditions for a class of constrained sweeping processes with nonsmooth perturbations. Optimal control problems for sweeping processes have been recently recognized among the most interesting and challenging problems in modern control theory for discontinuous differential inclusions with irregular dynamics and implicit state constrained, while deriving necessary optimality conditions for their local minimizers have been significantly based on the smoothness of controlled dynamic pertu…
We investigate constrained optimal control problems for linear stochastic dynamical systems evolving in discrete time. We consider minimization of an expected value cost over a finite horizon. Hard constraints are introduced first, and then reformulated in terms of probabilistic constraints. It is shown that, for a suitable parametrization of the control policy, a wide class of the resulting optimization problems are convex, or admit reasonable convex approximations.
The best techniques for the constrained maximum-entropy sampling problem, a discrete-optimization problem arising in the design of experiments, are via a variety of concave continuous relaxations of the objective function. A standard bound-enhancement technique in this context is scaling. We extend this technique to generalized scaling, we give mathematical results aimed at supporting algorithmic methods for computing optimal generalized scalings, and we give computational results demonstrating the usefulness of generalized scaling on benchmark problem instances.
We propose a sampling-based trajectory optimization methodology for constrained problems. We extend recent works on stochastic search to deal with box control constraints,as well as nonlinear state constraints for discrete dynamical systems. Regarding the former, our strategy is to optimize over truncated parameterized distributions on control inputs. Furthermore, we show how non-smooth penalty functions can be incorporated into our framework to handle state constraints. Simulations on cartpole and quadcopter show that our approach outperforms previous methods on constrained sampling-based opt…
In this paper, a self-triggered control scheme for constrained discrete-time control systems is presented. The key idea of our approach is to construct a transition system or a graph structure from a collection of polyhedral sets, which are generated based on the notion of {set-invariance theory}. The inter-event time steps are then determined through a standard graph search algorithm to obtain the minimal total cost to a terminal state. The proposed strategy is illustrated through a numerical example.
This paper presents a state and state-input constrained variant of the discrete-time iterative Linear Quadratic Regulator (iLQR) algorithm, with linear time-complexity in the number of time steps. The approach is based on a projection of the control input onto the nullspace of the linearized constraints. We derive a fully constraint-compliant feedforward-feedback control update rule, for which we can solve efficiently with Riccati-style difference equations. We assume that the relative degree of all constraints in the discrete-time system model is equal to one, which often holds for robotics p…
In this paper we discuss how to decompose the constrained generalized discrete-time algebraic Riccati equation arising in optimal control and optimal filtering problems into two parts corresponding to an additive decomposition X=X0+D of each solution X: The first part is an explicit expression of the addend X0 which is common to all solutions, and does not depend on the particular X. The second part can be either a reduced-order discrete-time regular algebraic Riccati equation whose associated closed-loop matrix is non-singular, or a symmetric Stein equation.
We consider chance-constrained problems with discrete random distribution. We aim for problems with a large number of scenarios. We propose a novel method based on the stochastic gradient descent method which performs updates of the decision variable based only on considering a few scenarios. We modify it to handle the non-separable objective. Complexity analysis and a comparison with the standard (batch) gradient descent method is provided. We give three examples with non-convex data and show that our method provides a good solution fast even when the number of scenarios is large.
This paper deals with optimal control problems described by a controlled version of Moreau's sweeping process governed by convex polyhedra, where measurable control actions enter additive perturbations. This class of problems, which addresses unbounded discontinuous differential inclusions with intrinsic state constraints, is truly challenging and underinvestigated in control theory while being highly important for various applications. To attack such problems with constrained measurable controls, we develop a refined method of discrete approximations with establishing its well-posedness and s…
In this paper, we study a mean-variance optimization problem in an infinite horizon discrete time discounted Markov decision process (MDP). The objective is to minimize the variance of system rewards with the constraint of mean performance. Different from most of works in the literature which require the mean performance already achieve optimum, we can let the mean discounted performance equal any constant. The difficulty of this problem is caused by the quadratic form of the variance function which makes the variance minimization problem not a standard MDP. By proving the decomposable structu…
We address the optimal covariance steering (OCS) problem for stochastic discrete linear systems with additive Gaussian noise under state chance constraints and input hard constraints. Because the system state can be unbounded due to the unbounded noise, the state constraints are formulated as probabilistic (chance) constraints, i.e., the maximum probability of constraint violation is constrained. In contrast, because it is hard to interpret the appropriate control action when the control command violates the constraints, probabilistically formulating the control constraints are difficult, and …
We present a novel convex relaxation and a corresponding inference algorithm for the non-binary discrete tomography problem, that is, reconstructing discrete-valued images from few linear measurements. In contrast to state of the art approaches that split the problem into a continuous reconstruction problem for the linear measurement constraints and a discrete labeling problem to enforce discrete-valued reconstructions, we propose a joint formulation that addresses both problems simultaneously, resulting in a tighter convex relaxation. For this purpose a constrained graphical model is set up a…
