In this paper, we consider a varying terminal time structure for the stochastic optimal control problem under state constraints, in which the terminal time varies with the mean value of the state. In this new stochastic optimal control system, the control domain does not need to be convex and the diffusion coefficient contains the control variable. To overcome the difficulty in the proof of the related Pontryagin's stochastic maximum principle, we develop asymptotic first- and second-order adjoint equations for the varying terminal time, and then establish its variational equation. In the end, two examples are given to verify the main results of this study.
In this study, we propose a varying terminal time structure for the optimal control problem under state constraints, in which the terminal time follows the varying of the control via the constrained condition. Focusing on this new optimal control problem, we investigate a novel stochastic maximum principle, which differs from the traditional optimal control problem under state constraints. The optimal pair of the optimal control model can be verified via this new stochastic maximum principle.
To improve the efficient frontier of the classical mean-variance model in continuous time, we propose a varying terminal time mean-variance model with a constraint on the mean value of the portfolio asset, which moves with the varying terminal time. Using the embedding technique from stochastic optimal control in continuous time and varying the terminal time, we determine an optimal strategy and related deterministic terminal time for the model. Our results suggest that doing so for an investment plan requires minimizing the variance with a varying terminal time.
Stochastic optimal control problems with constraints on the probability distribution of the final output are considered. Necessary conditions for optimality in the form of a coupled system of partial differential equations involving a forward Fokker-Planck equation and a backward Hamilton-Jacobi-Bellman equation are proved using convex duality techniques.
Model predictive control (MPC) schemes are commonly designed with fixed, i.e., time-invariant, horizon length and cost functions. If no stabilizing terminal ingredients are used, stability can be guaranteed via a sufficiently long horizon. A suboptimality index can be derived that gives bounds on the performance of the MPC law over an infinite-horizon (IH). While for time-invariant schemes such index can be computed offline, less attention has been paid to time-varying strategies with adapting cost function which can be found, e.g., in learning-based optimal control. This work addresses the pe…
We consider a stochastic control problem, where the control domain is convex and the system is governed by a nonlinear backward stochastic differential equation. With a L1 terminal data, we derive necessary optimality conditions in the form of stochastic maximum principle.
Devising efficient algorithms that track the optimizers of continuously varying convex optimization problems is key in many applications. A possible strategy is to sample the time-varying problem at constant rate and solve the resulting time-invariant problem. This can be too computationally burdensome in many scenarios. An alternative strategy is to set up an iterative algorithm that generates a sequence of approximate optimizers, which are refined every time a new sampled time-invariant problem is available by one iteration of the algorithm. These types of algorithms are called running. A ma…
We develop a novel convex parametrization of integral quadratic constraints with a terminal cost for subdifferentials of convex functions, involving general O'Shea-Zames-Falb multipliers. We show the benefit of our results for the reduction of conservatism of existing techniques, and sketch applications to the analysis of optimization algorithms or the stability analysis of neural network controllers. The development is prepared by providing a novel link between the convex integrability of a multivariable mapping and dissipativity theory.
A stochastic linear quadratic (LQ) optimal control problem with a pointwise linear equality constraint on the terminal state is considered. A strong Lagrangian duality theorem is proved under a uniform convexity condition on the cost functional and a surjectivity condition on the linear constraint mapping. Based on the Lagrangian duality, two approaches are proposed to solve the constrained stochastic LQ problem. First, a theoretical method is given to construct the closed-form solution by the strong duality. Second, an iterative algorithm, called augmented Lagrangian method (ALM), is proposed…
University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of California, Santa Cruz（加州大学圣克鲁兹分校）
In this work, we analyze the properties of the solution to the covariance steering problem for discrete time Gaussian linear systems with a squared Wasserstein distance terminal cost. In our previous work, we have shown that by utilizing the state feedback control policy parametrization, this stochastic optimal control problem can be associated with a difference of convex functions program. Here, we revisit the same covariance control problem but this time we focus on the analysis of the problem. Specifically, we establish the existence of solutions to the optimization problem and derive the f…
The second part of our study is devoted to an analysis of the exactness of penalty functions for optimal control problems with terminal and pointwise state constraints. We demonstrate that with the use of the exact penalty function method one can reduce fixed-endpoint problems for linear time-varying systems and linear evolution equations with convex constraints on the control inputs to completely equivalent free-endpoint optimal control problems, if the terminal state belongs to the relative interior of the reachable set. In the nonlinear case, we prove that a local reduction of fixed-endpoin…
We consider cooperative multi-agent consensus optimization problems over both static and time-varying communication networks, where only local communications are allowed. The objective is to minimize the sum of agent-specific possibly non-smooth composite convex functions over agent-specific private conic constraint sets; hence, the optimal consensus decision should lie in the intersection of these private sets. Assuming the sum function is strongly convex, we provide convergence rates in suboptimality, infeasibility and consensus violation; examine the effect of underlying network topology on…
This note studies the distributed non-convex optimization problem with non-smooth regularization, which has wide applications in decentralized learning, estimation and control. The objective function is the sum of different local objective functions, which consist of differentiable (possibly non-convex) cost functions and non-smooth convex functions. This paper presents a distributed proximal gradient algorithm for the non-smooth non-convex optimization problem over time-varying multi-agent networks. Each agent updates local variable estimate by the multi-step consensus operator and the proxim…
In this paper, a time-varying distributed convex optimization problem is studied for continuous-time multi-agent systems. Control algorithms are designed for the cases of single-integrator and double-integrator dynamics. Two discontinuous algorithms based on the signum function are proposed to solve the problem in each case. Then in the case of double-integrator dynamics, two continuous algorithms based on, respectively, a time-varying and a fixed boundary layer are proposed as continuous approximations of the signum function. Also, to account for inter-agent collision for physical agents, a d…
This paper considers a distributed convex optimization problem over a time-varying multi-agent network, where each agent has its own decision variables that should be set so as to minimize its individual objective subject to local constraints and global coupling equality constraints. Over directed graphs, a distributed algorithm is proposed that incorporates the push-sum protocol into dual subgradient methods. Under the convexity assumption, the optimality of primal and dual variables, and constraint violations is first established. Then the explicit convergence rates of the proposed algorithm…
In the large collection of existing distributed algorithms for convex multi-agent optimization, only a handful of them provide convergence rate guarantees on agent networks with time-varying topologies, which, however, restrict the problem to be unconstrained. Motivated by this, we develop a family of distributed Fenchel dual gradient methods for solving constrained, strongly convex but not necessarily smooth multi-agent optimization problems over time-varying undirected networks. The proposed algorithms are constructed based on the application of weighted gradient methods to the Fenchel dual …
In this paper, we develop an interior-point method for solving a class of convex optimization problems with time-varying objective and constraint functions. Using log-barrier penalty functions, we propose a continuous-time dynamical system for tracking the (time-varying) optimal solution with an asymptotically vanishing error. This dynamical system is composed of two terms: (i) a correction term consisting of a continuous-time version of Newton's method, and (ii) a prediction term able to track the drift of the optimal solution by taking into account the time-varying nature of the objective an…
For discrete time nonlinear systems satisfying an exponential or finite time controllability assumption, we present an analytical formula for a suboptimality estimate for model predictive control schemes without stabilizing terminal constraints. Based on our formula, we perform a detailed analysis of the impact of the optimization horizon and the possibly time varying control horizon on stability and performance of the closed loop.
Devising efficient algorithms to solve continuously-varying strongly convex optimization programs is key in many applications, from control systems to signal processing and machine learning. In this context, solving means to find and track the optimizer trajectory of the continuously-varying convex optimization program. Recently, a novel prediction-correction methodology has been put forward to set up iterative algorithms that sample the continuously-varying optimization program at discrete time steps and perform a limited amount of computations to correct their approximate optimizer with the …
We consider a class of stochastic optimal control problems for discrete-time stochastic linear systems which seek for control policies that will steer the probability distribution of the terminal state of the system close to a desired Gaussian distribution. In our problem formulation, the closeness between the terminal state distribution and the desired (goal) distribution is measured in terms of the squared Wasserstein distance which is associated with a corresponding terminal cost term. We recast the stochastic optimal control problem as a finite-dimensional nonlinear program and we show tha…
In this paper, a distributed convex optimization problem with swarm tracking behavior is studied for continuous-time multi-agent systems. The agents' task is to drive their center to track an optimal trajectory which minimizes the sum of local time-varying cost functions through local interaction, while maintaining connectivity and avoiding inter-agent collision. Each local cost function is only known to an individual agent and the team's optimal solution is time-varying. Here two cases are considered, single-integrator dynamics and double-integrator dynamics. For each case, a distributed conv…
For nonlinear discrete time systems satisfying a controllability condition, we present a stability condition for model predictive control without stabilizing terminal constraints or costs. The condition is given in terms of an analytical formula which can be employed in order to determine a prediction horizon length for which asymptotic stability or a performance guarantee is ensured. Based on this formula a sensitivity analysis with respect to the prediction and the possibly time varying control horizon is carried out.
This paper presents a decentralized algorithm for a team of agents to track time-varying fixed points that are the solutions to time-varying convex optimization problems. The algorithm is first-order, and it allows for total asynchrony in the communications and computations of all agents, i.e., all such operations can occur with arbitrary timing and arbitrary (finite) delays. Convergence rates are computed in terms of the communications and computations that agents execute, without specifying when they must occur. These rates apply to convergence to the minimum of each individual objective fun…
We investigate a stochastic optimal control problem where the controlled system is depicted as a stochastic differential delayed equation; however, at the terminal time, the state is constrained in a convex set. We firstly introduce an equivalent backward delayed system depicted as a time-delayed backward stochastic differential equation. Then a stochastic maximum principle is obtained by virtue of Ekeland's variational principle. Finally, applications to a state constrained stochastic delayed linear-quadratic control model and a production-consumption choice problem are studied to illustrate …
In this paper, we propose a self-triggered algorithm to solve a class of convex optimization problems with time-varying objective functions. It is known that the trajectory of the optimal solution can be asymptotically tracked by a continuous-time state update law. Unfortunately, implementing this requires continuous evaluation of the gradient and the inverse Hessian of the objective function which is not amenable to digital implementation. Alternatively, we draw inspiration from self-triggered control to propose a strategy that autonomously adapts the times at which it makes computations abou…
This paper considers online convex optimization with time-varying constraint functions. Specifically, we have a sequence of convex objective functions $\{f_t(x)\}_{t=0}^{\infty}$ and convex constraint functions $\{g_{t,i}(x)\}_{t=0}^{\infty}$ for $i \in \{1, ..., k\}$. The functions are gradually revealed over time. For a given $ε>0$, the goal is to choose points $x_t$ every step $t$, without knowing the $f_t$ and $g_{t,i}$ functions on that step, to achieve a time average at most $ε$ worse than the best fixed-decision that could be chosen with hindsight, subject to the time average of the con…
We investigate a distributed optimization problem over a cooperative multi-agent time-varying network, where each agent has its own decision variables that should be set so as to minimize its individual objective subject to local constraints and global coupling constraints. Based on push-sum protocol and dual decomposition, we design a distributed regularized dual gradient algorithm to solve this problem, in which the algorithm is implemented in time-varying directed graphs only requiring the column stochasticity of communication matrices. By augmenting the corresponding Lagrangian function wi…
A Learning Model Predictive Controller (LMPC) for linear system in presented. The proposed controller is an extension of the LMPC [1] and it aims to decrease the computational burden. The control scheme is reference-free and is able to improve its performance by learning from previous iterations. A convex safe set and a terminal cost function are used in order to guarantee recursive feasibility and non-increasing performance at each iteration. The paper presents the control design approach, and shows how to recursively construct the convex terminal set and the terminal cost from state and inpu…
This paper addresses the design and analysis of feedback-based online algorithms to control systems or networked systems based on performance objectives and engineering constraints that may evolve over time. The emerging time-varying convex optimization formalism is leveraged to model optimal operational trajectories of the systems, as well as explicit local and network-level operational constraints. Departing from existing batch and feed-forward optimization approaches, the design of the algorithms capitalizes on an online implementation of primal-dual projected-gradient methods; the gradient…
The auxiliary problem principle of augmented Lagrangian (APP-AL), proposed by Cohen and Zhu (1984), aims to find the solution of a constrained optimization problem through a sequence of auxiliary problems involving augmented Lagrangian. The merits of this approach are two folds. First, the core function is usually separable, which makes the subproblems at each step decomposable and particularly attractive for parallel computing. Second, the choice of the core function is quite flexible. Consequently, by carefully specifying this function, APP-AL may reduce to some standard optimization algorit…
In this paper we consider a distributed convex optimization problem over time-varying networks. We propose a dual method that converges R-linearly to the optimal point given that the agents' objective functions are strongly convex and have Lipschitz continuous gradients. The proposed method requires half the amount of variable exchanges per iterate than methods based on DIGing, and yields improved practical performance as empirically demonstrated.
In this paper we consider a distributed convex optimization problem over time-varying undirected networks. We propose a dual method, primarily averaged network dual ascent (PANDA), that is proven to converge R-linearly to the optimal point given that the agents objective functions are strongly convex and have Lipschitz continuous gradients. Like dual decomposition, PANDA requires half the amount of variable exchanges per iterate of methods based on DIGing, and can provide with practical improved performance as empirically demonstrated.
This paper presents a new concept of controlled dissipativity as an extension of the standard dissipativity property to systems with parameter-varying storage functions under the framework of economic model predictive control (EMPC). Based on this concept, two EMPC controllers, integrated with the dissipation inequality constraints rendering the storage function parameters as decision variables, are formulated and the associated recursive feasibility is ensured. Then, the asymptotic convergence to an optimal equilibrium in closed-loop, without requiring the standard dissipativity assumption, i…
Hamburg University of Technology（汉堡工业大学）；UC Berkeley（加州大学伯克利分校）
The goal of this thesis is to design a learning model predictive controller (LMPC) that allows multiple agents to race competitively on a predefined race track in real-time. This thesis addresses two major shortcomings in the already existing single-agent formulation. Previously, the agent determines a locally optimal trajectory but does not explore the state space, which may be necessary for overtaking maneuvers. Additionally, obstacle avoidance for LMPC has been achieved in the past by using a non-convex terminal set, which increases the complexity for determining a solution to the optimizat…
Adaptive Horizon Model Predictive Control (AHMPC) is a scheme for varying as needed the horizon length of Model Predictive Control (MPC). Its goal is to achieve stabilization with horizons as small as possible so that MPC can be used on faster or more complicated dynamic processes. Beside the standard requirements of MPC including a terminal cost that is a control Lyapunov function, AHMPC requires a terminal feedback that turns the control Lyapunov function into a standard Lyapunov function in some domain around the operating point. But this domain need not be known explicitly. MPC does not co…
In this article we solve the problem of maximizing the expected utility of future consumption and terminal wealth to determine the optimal pension or life-cycle fund strategy for a cohort of pension fund investors. The setup is strongly related to a DC pension plan where additionally (individual) consumption is taken into account. The consumption rate is subject to a time-varying minimum level and terminal wealth is subject to a terminal floor. Moreover, the preference between consumption and terminal wealth as well as the intertemporal coefficient of risk aversion are time-varying and therefo…
In this paper we present a Learning Model Predictive Control (LMPC) strategy for linear and nonlinear time optimal control problems. Our work builds on existing LMPC methodologies and it guarantees finite time convergence properties for the closed-loop system. We show how to construct a time varying safe set and terminal cost function using closed-loop data. The resulting LMPC policy is time varying and it guarantees recursive constraint satisfaction and non-decreasing performance. Computational efficiency is obtained by convexifing the safe set and terminal cost function. We demonstrate that,…
This paper considers the problem of distributed bandit online convex optimization with time-varying coupled inequality constraints. This problem can be defined as a repeated game between a group of learners and an adversary. The learners attempt to minimize a sequence of global loss functions and at the same time satisfy a sequence of coupled constraint functions. The global loss and the coupled constraint functions are the sum of local convex loss and constraint functions, respectively, which are adaptively generated by the adversary. The local loss and constraint functions are revealed in a …
In this article, two methods for solving mean-field type optimal control problems are proposed and investigated. The two methods are iterative methods: at each iteration, a Hamilton-Jacobi-Bellman equation is solved, for a terminal condition obtained by linearizing the cost function. The terminal condition is updated by solving a Fokker-Planck equation. The first method can be seen as a gradient method and uses in an essential manner the convexity of the set of probability distributions. A convergence result for this method is provided. The second method incorporates a penalization term and pr…
In this paper we consider distributed convex optimization over time-varying undirected graphs. We propose a linearized version of primarily averaged network dual ascent (PANDA) while requiring less computational costs. The proposed method, economic primarily averaged network dual ascent (Eco-PANDA), provably converges at R-linear rate to the optimal point given that the agents' objective functions are strongly convex and have Lipschitz continuous gradients. Therefore, the method is competitive, in terms of type of rate, with both DIGing and PANDA. The proposed method halves the communication c…
