This paper addresses the problem of steering a discrete-time linear dynamical system from an initial Gaussian distribution to a final distribution in a game-theoretic setting. One of the two players strives to minimize a quadratic payoff, while at the same time tries to meet a given mean and covariance constraint at the final time-step. The other player maximizes the same payoff, but it is assumed to be indifferent to the terminal constraint. At first, the unconstrained version of the game is examined, and the necessary conditions for the existence of a saddle point are obtained. We then show that obtaining a solution for the one-sided constrained dynamic game is not guaranteed, and subsequently the players' best responses are analyzed. Finally, we propose to numerically solve the problem of steering the distribution under adversarial scenarios using the Jacobi iteration method. The problem of guiding a missile during the endgame is chosen to analyze the proposed approach. A numerical simulation corresponding to the case where the terminal distribution is not achieved is also included, and discuss the necessary conditions to meet the terminal constraint.
This work investigates the finite-horizon optimal covariance steering problem for discrete-time linear systems subject to both additive and multiplicative uncertainties as well as state and input chance constraints. In particular, a tractable convex approximation of the optimal covariance steering problem is developed by tightening the chance constraints and by introducing a suitable change of variables. The solution of the convex approximation is shown to be a valid (albeit potentially suboptimal) solution to the original chance-constrained covariance steering problem.
This work considers the optimal covariance steering problem for systems subject to both additive noise and uncertain parameters which may enter multiplicatively with the state and the control. The unknown parameters are modeled as a constant random variable sampled from a distribution with known moments. The optimal covariance steering problem is formulated using a moment-based representation of the system dynamics, which includes dependence between the unknown parameters and future states, and is solved using sequential convex programming. The proposed approach is demonstrated numerically usi…
We consider the covariance steering problem for nonlinear control-affine systems. Our objective is to find an optimal control strategy to steer the state of a system from an initial distribution to a target one whose mean and covariance are given. Due to the nonlinearity, the existing techniques for linear covariance steering problems are not directly applicable. By leveraging the celebrated Girsanov theorem, we formulate the problem into an optimization over the space path distributions. We then adopt a generalized proximal gradient algorithm to solve this optimization, where each update requ…
This article extends the optimal covariance steering (CS) problem for discrete time linear stochastic systems modeled using moment-based ambiguity sets. To hedge against the uncertainty in the state distributions while performing covariance steering, distributionally robust risk constraints are employed during the optimal allocation of the risk. Specifically, a distributionally robust iterative risk allocation (DR-IRA) formalism is used to solve the optimal risk allocation problem for the CS problem using a two-stage approach. The upper-stage of DR-IRA is a convex problem that optimizes the ri…
In this paper we study the finite-horizon optimal covariance steering problem for a continuous-time linear stochastic system subject to both additive and multiplicative noise. The noise can be continuous or it may contain jumps. Additive noise does not depend on the state or the control, whereas multiplicative noise has a magnitude proportional to the current state. The cost is assumed to be quadratic in both the state and the control. First, the controllability of the state covariance is established under mild assumptions. Then, the optimal control for steering the covariance is provided. Las…
In this work, we consider the problem of steering the first two moments of the uncertain state of an unknown discrete-time stochastic nonlinear system to a given terminal distribution in finite time. Toward that goal, first, a non-parametric predictive model is learned from a set of available training data points using stochastic variational Gaussian process regression: a powerful and scalable machine learning tool for learning distributions over arbitrary nonlinear functions. Second, we formulate a tractable nonlinear covariance steering algorithm that utilizes the Gaussian process predictive…
In this work, we consider the problem of steering the first two moments of the uncertain state of a discrete time nonlinear stochastic system to prescribed goal quantities at a given final time. In principle, the latter problem can be formulated as a density tracking problem, which seeks for a feedback policy that will keep the probability density function of the state of the system close, in terms of an appropriate metric, to the goal density. The solution to the latter infinite-dimensional problem can be, however, a complex and computationally expensive task. Instead, we propose a more tract…
In this paper, we study the covariance steering (CS) problem for discrete-time linear systems subject to multiplicative and additive noise. Specifically, we consider two variants of the so-called CS problem. The goal of the first problem, which is called the exact CS problem, is to steer the mean and the covariance of the state process to their desired values in finite time. In the second one, which is called the ``relaxed'' CS problem, the covariance assignment constraint is relaxed into a positive semi-definite constraint. We show that the relaxed CS problem can be cast as an equivalent conv…
Continuous-Time Covariance Steering with Common Free-Final Time: Finite-Horizon Solutions and Infinite-Horizon Limits
Daniel Guggenheim School of Aerospace, Georgia Institute of Technology（佐治亚理工学院）；Department of Electrical and Computer Engineering, FAMU-FSU College of Engineering（FAMU-FSU工程学院）
The problem of optimizing affine feedback laws that explicitly steer the mean and covariance of an uncertain system state in the presence of a Gaussian random field is considered. Spatially-dependent disturbances are successively approximated with respect to a nominal trajectory by a sequence of jointly Gaussian random vectors. Sequential updates to the nominal control inputs are computed via convex optimization that includes the effect of affine state feedback, the perturbing effects of spatial disturbances, and chance constraints on the closed-loop state and control. The developed method is …
In this paper, we present a new trajectory optimization algorithm for stochastic linear systems which combines Model Predictive Path Integral (MPPI) control with Constrained Covariance Steering (CSS) to achieve high performance with safety guarantees (robustness). Although MPPI can be used to solve complex nonlinear trajectory optimization problems, it may not always handle constraints effectively and its performance may degrade in the presence of unmodeled disturbances. By contrast, CCS can handle probabilistic state and / or input constraints (e.g., chance constraints) and also steer the sta…
The goal of this paper is to address finite-horizon minimum variance and covariance steering problems for discrete-time stochastic (Gaussian) linear systems. On the one hand, the minimum variance problem seeks for a control policy that will steer the state mean of an uncertain system to a prescribed quantity while minimizing the trace of its terminal state covariance (or variance). On the other hand, the covariance steering problem seeks for a control policy that will steer the covariance of the terminal state to a prescribed positive definite matrix. We propose a solution approach that relies…
In this paper, we present new results on the covariance steering problem with Wasserstein distance terminal cost. We show that the state history feedback control policy parametrization, which has been used before to solve this class of problems, requires an unnecessarily large number of variables and can be replaced by a randomized state feedback policy which leads to more tractable problem formulations without any performance loss. In particular, we show that under the latter policy, the problem can be equivalently formulated as a semi-definite program (SDP) which is in sharp contrast with ou…
This paper considers the problem of steering the state distribution of a nonlinear stochastic system from an initial Gaussian to a terminal distribution with a specified mean and covariance, subject to probabilistic path constraints. An algorithm is developed to solve this problem by iteratively solving an approximate linearized problem as a convex program. This method, which we call iterative covariance steering (iCS), is numerically demonstrated by controlling a double integrator with quadratic drag force subject to additive Brownian noise while satisfying probabilistic path constraints.
In this paper, we present a new control policy parametrization for the finite-horizon covariance steering problem for discrete-time Gaussian linear systems (DTGLS) which can reduce the latter stochastic optimal control problem to a tractable optimization problem. The covariance steering problem seeks for a feedback control policy that will steer the state covariance of a DTGLS to a desired positive definite matrix in finite time. We consider two different formulations of the covariance steering problem, one with hard terminal LMI constraints (hard-constrained covariance steering) and another o…
This work addresses the optimal covariance control problem for stochastic discrete-time linear time-varying systems subject to chance constraints. Covariance steering is a stochastic control problem to steer the system state Gaussian distribution to another Gaussian distribution while minimizing a cost function. To the best of our knowledge, covariance steering problems have never been discussed with probabilistic chance constraints although it is a natural extension. In this work, first we show that, unlike the case with no chance constraints, the covariance steering with chance constraints p…
This work develops a stochastic model predictive controller~(SMPC) for uncertain linear systems with additive Gaussian noise subject to state and control constraints. The proposed approach is based on the recently developed finite-horizon optimal covariance steering control theory, which steers the mean and the covariance of the system state to prescribed target values at a given terminal time. We call our approach covariance steering-based SMPC, or CS-SMPC. We show that the proposed approach has several advantages over traditional SMPC approaches in the literature. Specifically, it is shown t…
We consider a class of stochastic optimal control problems for discrete-time stochastic linear systems which seek for control policies that will steer the probability distribution of the terminal state of the system close to a desired Gaussian distribution. In our problem formulation, the closeness between the terminal state distribution and the desired (goal) distribution is measured in terms of the squared Wasserstein distance which is associated with a corresponding terminal cost term. We recast the stochastic optimal control problem as a finite-dimensional nonlinear program and we show tha…
We address the optimal covariance steering (OCS) problem for stochastic discrete linear systems with additive Gaussian noise under state chance constraints and input hard constraints. Because the system state can be unbounded due to the unbounded noise, the state constraints are formulated as probabilistic (chance) constraints, i.e., the maximum probability of constraint violation is constrained. In contrast, because it is hard to interpret the appropriate control action when the control command violates the constraints, probabilistically formulating the control constraints are difficult, and …
University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of California, Santa Cruz（加州大学圣克鲁兹分校）
In this work, we analyze the properties of the solution to the covariance steering problem for discrete time Gaussian linear systems with a squared Wasserstein distance terminal cost. In our previous work, we have shown that by utilizing the state feedback control policy parametrization, this stochastic optimal control problem can be associated with a difference of convex functions program. Here, we revisit the same covariance control problem but this time we focus on the analysis of the problem. Specifically, we establish the existence of solutions to the optimization problem and derive the f…
This paper extends the optimal covariance steering problem for linear stochastic systems subject to chance constraints to account for optimal risk allocation. Previous works have assumed a uniform risk allocation to cast the optimal control problem as a semi-definite program (SDP), which can be solved efficiently using standard SDP solvers. We adopt an Iterative Risk Allocation (IRA) formalism, which uses a two-stage approach to solve the optimal risk allocation problem for covariance steering. The upper-stage of IRA optimizes the risk, which is proved to be a convex problem, while the lower-s…
This work addresses the problem of vehicle path planning in the presence of obstacles and uncertainties, which is a fundamental problem in robotics. While many path planning algorithms have been proposed for decades, many of them have dealt with only deterministic environments or only open-loop uncertainty, i.e., the uncertainty of the system state is not controlled and, typically, increases with time due to exogenous disturbances, which leads to the design of potentially conservative nominal paths. In order to deal with disturbances and reduce uncertainty, generally, a lower-level feedback co…
Operator Splitting Covariance Steering for Safe Stochastic Nonlinear Control
School of Aerospace Engineering, Georgia Institute of Technology（航空航天工程学院，佐治亚理工学院）；School of Electrical and Computer Engineering, Georgia Institute of Technology（电气与计算机工程学院，佐治亚理工学院）
We consider the problem of steering a linear stochastic system between two end-point degenerate Gaussian distributions in finite time. This accounts for those situations in which some but not all of the state entries are uncertain at the initial, t = 0, and final time, t = T . This problem entails non-trivial technical challenges as the singularity of terminal state-covariance causes the control to grow unbounded at the final time T. Consequently, the entropic interpolation (Schroedinger Bridge) is provided by a diffusion process which is not finite-energy, thereby placing this case outside of…
We consider the problem of steering, via output feedback, the state distribution of a discrete-time, linear stochastic system from an initial Gaussian distribution to a terminal Gaussian distribution with prescribed mean and maximum covariance, subject to probabilistic path constraints on the state. The filtered state is obtained via a Kalman filter, and the problem is formulated as a deterministic convex program in terms of the distribution of the filtered state. We observe that, in the presence of constraints on the state covariance, and in contrast to classical Linear Quadratic Gaussian (LQ…
In this paper, we define and solve the Inverse Stochastic Optimal Control (ISOC) problem of the linear-quadratic Gaussian (LQG) and the linear-quadratic sensorimotor (LQS) control model. These Stochastic Optimal Control (SOC) models are state-of-the-art approaches describing human movements. The LQG ISOC problem consists of finding the unknown weighting matrices of the quadratic cost function and the covariance matrices of the additive Gaussian noise processes based on ground truth trajectories observed from the human in practice. The LQS ISOC problem aims at additionally finding the covarianc…
We develop a new approach to robust adaptive beamforming in the presence of signal steering vector errors. Since the signal steering vector is known imprecisely, its presumed (prior) value is used to find a more accurate estimate of the actual steering vector, which then is used for obtaining the optimal beamforming weight vector. The objective for finding such an estimate of the actual signal steering vector is the maximization of the beamformer output power, while the constraints are the normalization condition and the requirement that the estimate of the steering vector does not converge to…
Markov Kernels, Distances and Optimal Control: A Parable of Linear Quadratic Non-Gaussian Distribution Steering
Department of Applied Mathematics, University of California, Santa Cruz（应用数学系，加州大学圣克鲁兹分校）；Department of Aerospace Engineering, Iowa State University（航空航天工程系，爱荷华州立大学）；Theoretical Division, Los Alamos National Laboratory（理论 division，洛斯阿拉莫斯国家实验室）
We propose an approach to use the state covariance of linear systems to track time-varying covariance matrices of non-stationary time series. Following concepts from Riemmanian geometry, we investigate three types of covariance paths obtained by using different quadratic regularizations of system matrices. The first quadratic form induces the geodesics based on the Bures-Wasserstein metric from optimal mass transport theory and quantum mechanics. The second type of quadratic form leads to the geodesics based on the Fisher-Rao metric from information geometry. In the process, we introduce a flu…
