A system of a parabolic partial differential equation coupled with ordinary differential inclusions that arises from a closed-loop control problem for a thermodynamic process governed by the Allen-Cahn diffusion reaction model is studied. A feedback law for the closed-loop control is proposed and implemented in the case of a finite number of control devices located inside the process domain basing on the process dynamics observed at a finite number of measurement points. The existence of solutions to the discussed system of differential equations is proved with the use of a generalization of the Kakutani fixed point theorem.
This paper is concerned with a bilinear control problem for enhancing convection-cooling via an incompressible velocity field. Both optimal open-loop control and closed-loop feedback control designs are addressed. First and second order optimality conditions for characterizing the optimal solution are discussed. In particular, the method of instantaneous control is applied to establish the feedback laws. Moreover, the construction of feedback laws is also investigated by directly utilizing the optimality system with appropriate numerical discretization schemes. Computationally, it is much easi…
We propose a virtual clinical trial for assessing the safety and efficacy of closed-loop diabetes treatments prior to an actual clinical trial. Such virtual trials enable rapid and risk-free pretrial testing of algorithms, and they can be used to compare different treatment variations for large and diverse populations. The participants are represented by multiple mathematical models, consisting of stochastic differential equations, and we use Monte Carlo closed-loop simulations to compute detailed statistics of the closed-loop treatments. We implement the virtual clinical trial using high-perf…
This paper proposes an adaptive neuro-fuzzy framework to improve drug infusion rate in closed-loop control of anesthesia. The proposed controller provides a sub-optimal propofol administration rate as input to reach the desired bispectral index, which is the output of the system, in both induction and maintenance phases. In this controller, a critic agent assesses the plant output and produces a reinforcement signal to adapt the controller parameters and minimize the propofol administration rate. The controller is applied to a conventional pharmacokinetic-pharmacodynamics model of anesthesia t…
In this paper, the stabilization problem with closed-loop domain of attraction (DOA) enlargement for discrete-time general nonlinear plants is solved. First, a sufficient condition for asymptotic stabilization and estimation of the closed-loop DOA is given. It shows that, for a given Lyapunov function, the negative-definite and invariant set in the state-control space is a stabilizing controller set and its projection along the control space to the state space can be an estimate of the closed-loop DOA. Then, an algorithm is proposed to approximate the negative-definite and invariant set for th…
In this article, the definition on the control ability, and the relation between the open-loop control ability and the closed-loop performance are studied systematically for the linear dynamical systems. Firstly, to define and compare rationally the state control ability between the different controlled plants or one controlled plant with the different system parameters, the normalization of the input variables, the state variables, the system mdeols are discussed. With the help of the normalization, the state control ability with the time attribute can be difined under the unit input constrai…
In this paper, we consider a linear quadratic stochastic two-person nonzero-sum differential game. Open-loop and closed-loop Nash equilibria are introduced. The existence of the former is characterized by the solvability of a system of forward-backward stochastic differential equations, and that of the latter is characterized by the solvability of a system of coupled symmetric Riccati differential equations. Sometimes, open-loop Nash equilibria admit a closed-loop representation, via the solution to a system of non-symmetric Riccati equations, which is different from the outcome of the closed-…
In this work, we propose a framework to learn feedback control policies with guarantees on closed-loop generalization and adversarial robustness. These policies are learned directly from expert demonstrations, contained in a dataset of state-control input pairs, without any prior knowledge of the task and system model. We use a Lipschitz-constrained loss minimization scheme to learn feedback policies with certified closed-loop robustness, wherein the Lipschitz constraint serves as a mechanism to tune the generalization performance and robustness to adversarial disturbances. Our analysis exploi…
This paper is concerned with mean-field stochastic linear-quadratic (MF-SLQ, for short) optimal control problems with deterministic coefficients. The notion of weak closed-loop optimal strategy is introduced. It is shown that the open-loop solvability is equivalent to the existence of a weak closed-loop optimal strategy. Moreover, when open-loop optimal controls exist, there is at least one of them admitting a state feedback representation, which is the outcome of a weak closed-loop optimal strategy. Finally, an example is presented to illustrate the procedure for finding weak closed-loop opti…
Recently it has been found that for a stochastic linear-quadratic optimal control problem (LQ problem, for short) in a finite horizon, open-loop solvability is strictly weaker than closed-loop solvability which is equivalent to the regular solvability of the corresponding Riccati equation. Therefore, when an LQ problem is merely open-loop solvable not closed-loop solvable, which is possible, the usual Riccati equation approach will fail to produce a state feedback representation of open-loop optimal controls. The objective of this paper is to introduce and investigate the notion of weak closed…
This paper is concerned with the closed-loop solvability of one kind of linear-quadratic Stackelberg stochastic differential game, where the coefficients are deterministic. The notion of the closed-loop solvability is introduced, which require to be independent of the initial state. The follower's problem is solved first, and the closed-loop optimal strategy is characterized by a Riccati equation, together with an adapted solution to a linear backward stochastic differential equation. Then the necessary conditions of the existence of the leader's nonanticipating closed-loop optimal strategy is…
Mitsubishi Electric Research Laboratories（三菱电机研究院）；École polytechnique fédérale de Lausanne（洛桑联邦理工学院）；Swiss Federal Laboratories for Materials Science and Technology（瑞士联邦材料科学与技术实验室）
We study the problem of performance optimization of closed-loop control systems with unmodeled dynamics. Bayesian optimization (BO) has been demonstrated effective for improving closed-loop performance by automatically tuning controller gains or reference setpoints in a model-free manner. However, BO methods have rarely been tested on dynamical systems with unmodeled constraints. In this paper, we propose a violation-aware BO algorithm (VABO) that optimizes closed-loop performance while simultaneously learning constraint-feasible solutions. Unlike classical constrained BO methods which allow a…
In this paper, we investigate the open-loop and weak closed-loop solvabilities of stochastic linear quadratic (LQ, for short) optimal control problem of Markovian regime switching system. Interestingly, these two solvabilities are equivalent. We first provide an alternative characterization of the open-loop solvability of the LQ problem using the perturbation approach. Then, we study the weak closed-loop solvability of the LQ problem of Markovian regime switching system, and establish the equivalent relationship between open-loop and weak closed-loop solvabilities. Finally, we present an examp…
This paper is concerned with the stochastic linear-quadratic optimal control problem with Poisson jumps. The coefficients in the state equation and the weighting matrices in the cost functional are all deterministic but are allowed indefinite. The notion of closed-loop strategies is introduced, and the optimal closed-loop strategy is characterized by a Riccati integral-differential equation and a backward stochastic differential equation with Poisson jumps.
This paper is concerned with a stochastic linear quadratic (LQ, for short) optimal control problem. The notions of open-loop and closed-loop solvabilities are introduced. A simple example shows that these two solvabilities are different. Closed-loop solvability is established by means of solvability of the corresponding Riccati equation, which is implied by the uniform convexity of the quadratic cost functional. Conditions ensuring the convexity of the cost functional are discussed, including the issue that how negative the control weighting matrix-valued function R(s) can be. Finiteness of th…
In this paper, we consider a linear quadratic stochastic two-person zero-sum differential game. The controls for both players are allowed to appear in both drift and diffusion of the state equation. The weighting matrices in the performance functional are not assumed to be definite/non-singular. A necessary and sufficient condition for the existence of a closed-loop saddle point is established in terms of the solvability of a Riccati differential equation with certain regularity. It is possible that the closed-loop saddle point fails to exist, and at the same time, the corresponding Riccati eq…
Large population games with interactions through controls and common noise: convergence results and equivalence between $open$--$loop$ and $closed$--$loop$ controls
This paper studies the closed-loop dynamics of linear systems under approximate model predictive control (MPC). More precisely, we consider MPC implementations based on a finite number of ADMM iterations per time-step. We first show that the closed-loop dynamics can be described based on a nonlinear augmented model. We then characterize an invariant set around the augmented origin, where the dynamics become linear. Finally, we investigate the performance of the approximate MPC for various choices of the ADMM parameters based on a comprehensive numerical benchmark.
We study the closed-loop solvability of a stochastic linear quadratic optimal control problem for systems governed by stochastic evolution equations. This solvability is established by means of solvability of the corresponding Riccati equation, which is implied by the uniform convexity of the quadratic cost functional. At last, conditions ensuring the uniform convexity of the cost functional are discussed.
An optimal control problem is studied for a linear mean-field stochastic differential equation with a quadratic cost functional. The coefficients and the weighting matrices in the cost functional are all assumed to be deterministic. Closed-loop strategies are introduced, which require to be independent of initial states; and such a nature makes it very useful and convenient in applications. In this paper, the existence of an optimal closed-loop strategy for the system (also called the closed-loop solvability of the problem) is characterized by the existence of a regular solution to the coupled…
In this paper, both dynamic mean-variance portfolio selection problems and dynamic variance hedging problems are discussed under non-Markovian framework. Explicit closed-loop equilibrium strategies of these problems are respectively obtained via a unified approach for the first time. Several new interesting facts arise in mean-variance problems with constant risk aversion. For example, it is shown that equilibrium strategies are still allowed to rely on initial wealth as long as risk-free return rate is random. In addition, the closed-loop equilibrium strategy and open-loop equilibrium strateg…
The paper deals with a class of time-inconsistent control problems for McKean-Vlasov dynamics. By solving a backward time-inconsistent Hamilton-Jacobi-Bellman (HJB for short) equation coupled with a forward distribution-dependent stochastic differential equation, we investigate the existence and uniqueness of a closed-loop equilibrium for such time-inconsistent distribution-dependent control problem. Moreover, a special case of semi-linear McKean-Vlasov dynamics with a quadratic-type cost functional is considered due to its special structure.
In this paper, the open-loop and closed-loop local and remote stochastic nonzero-sum game (LRSNG) problem is investigated. Different from previous works, the stochastic nonzero-sum game problem under consideration is essentially a special class of two-person nonzero-sum game problem, in which the information sets accessed by the two players are inconsistent. More specifically, both the local player and the remote player are involved in the system dynamics, and the information sets obtained by the two players are different, and each player is designed to minimize its own cost function. For the …
For an optimal control problem of an Itô's type stochastic differential equation, the control process could be taken as open-loop or closed-loop forms. In the standard literature, provided appropriate regularity, the value functions under these two types of controls are equal and are the unique (viscosity) solution to the corresponding (path-dependent) HJB equation. In this short note, we provide a counterexample in the path dependent setting showing that these value functions can be different in general.
This paper is concerned with a stochastic linear-quadratic optimal control problem in a finite time horizon, where the coefficients of the control system are allowed to be random, and the weighting matrices in the cost functional are allowed to be random and indefinite. It is shown, with a Hilbert space approach, that for the existence of an open-loop optimal control, the convexity of the cost functional (with respect to the control) is necessary; and the uniform convexity, which is slightly stronger, turns out to be sufficient, which also leads to the unique solvability of the associated stoc…
This paper investigates the stochastic linear quadratic (LQ, for short) optimal control problem of Markov regime switching system. The representation of the cost functional for the stochastic LQ optimal control problem of Markov regime switching system is derived using the technique of It{ô}'s formula. For the stochastic LQ optimal control problem of Markov regime switching system, we establish the equivalence between the open-loop (closed-loop) solvability and the existence of an adapted solution to the corresponding forward-backward stochastic differential equation with constraint (the exist…
