We consider continuous-time, finite-horizon, optimal quadratic control of semi-Markov jump linear systems (S-MJLS), and develop principled approximations through Markov-like representations for the holding-time distributions. We adopt a phase-type approximation for holding times, which is known to be consistent, and translates a S-MJLS into a specific MJLS with partially observable modes (MJLSPOM), where the modes in a cluster have the same dynamic, the same cost weighting matrices and the same control policy. For a general MJLSPOM, we give necessary and sufficient conditions for optimal (switched) linear controllers. When specialized to our particular MJLSPOM, we additionally establish the existence of optimal linear controller, as well as its optimality within the class of general controllers satisfying standard smoothness conditions. The known equivalence between phase-type distributions and positive linear systems allows to leverage existing modeling tools, but possibly with large computational costs. Motivated by this, we propose matrix exponential approximation of holding times, resulting in pseudo-MJLSPOM representation, i.e., where the transition rates could be negative. Such a representation is of relatively low order, and maintains the same optimality conditions as for the MJLSPOM representation, but could violate non-negativity of holding-time density functions. A two-step procedure consisting of a local pulling-up modification and a filtering technique is constructed to enforce non-negativity.
Robust Decentralized Stabilization of Markovian Jump Large-Scale Systems: A Neighboring Mode Dependent Control Approach
University of Science and Technology of China（中国科学技术大学）；University of New South Wales at the Australian Defence Force Academy（澳大利亚国防军学院新南威尔士大学工程学院与信息技术学院）
Jump Markov linear systems (JMLS) are a useful class which can be used to model processes which exhibit random changes in behavior during operation. This paper presents a numerically stable method for learning the parameters of jump Markov linear systems using the expectation-maximisation (EM) approach. The solution provided herein is a deterministic algorithm, and is not a Monte Carlo based technique. As a result, simulations show that when compared to alternative approaches, a more likely set of system parameters can be found within a fixed computation time, which better explain the observat…
In this paper, we study the stabilization of two interdependent Markov jump linear systems (MJLSs) with partial information, where the interdependency arises as the transition of the mode of one system depends on the states of the other system. First, we formulate a framework for the two interdependent MJLSs to capture the interactions between various entities in the system, where the modes of the system cannot be observed directly. Instead, a signal which contains information of the modes can be obtained. Then, depending on the scope of the available system state information (global or local)…
This paper investigates state feedback control laws for Markov jump linear systems with state and mode-observation delays. An assumption in this study is that the delay of mode observation obeys an exponential distribution. Also, we raise an unknown time-varying state delay applied in the composition of the state feedback controller. A method of remodeling the closed-loop system as a standard Markov jump linear system with state delay is shown. Furthermore, on the basis of this remodeling, several Linear Matrix Inequalities (LMI) for designing feedback gains for stabilization and mixed $H_2/H_…
In this work, we consider the problem of mode clustering in Markov jump models. This model class consists of multiple dynamical modes with a switching sequence that determines how the system switches between them over time. Under different active modes, the observations can have different characteristics. Given the observations only and without knowing the mode sequence, the goal is to cluster the modes based on their transition distributions in the Markov chain to find a reduced-rank Markov matrix that is embedded in the original Markov chain. Our approach involves mode sequence estimation, m…
This paper presents a Bayesian method for identification of jump Markov linear system parameters. A primary motivation is to provide accurate quantification of parameter uncertainty without relying on asymptotic in data-length arguments. To achieve this, the paper details a particle-Gibbs sampling approach that provides samples from the desired posterior distribution. These samples are produced by utilising a modified discrete particle filter and carefully chosen conjugate priors.
Switched systems are capable of modeling processes with underlying dynamics that may change abruptly over time. To achieve accurate modeling in practice, one may need a large number of modes, but this may in turn increase the model complexity drastically. Existing work on reducing system complexity mainly considers state space reduction, yet reducing the number of modes is less studied. In this work, we consider Markov jump linear systems (MJSs), a special class of switched systems where the active mode switches according to a Markov chain, and several issues associated with its mode complexit…
We present a learning model predictive control (MPC) scheme for chance-constrained Markov jump systems with unknown switching probabilities. Using samples of the underlying Markov chain, ambiguity sets of transition probabilities are estimated which include the true conditional probability distributions with high probability. These sets are updated online and used to formulate a time-varying, risk-averse optimal control problem. We prove recursive feasibility of the resulting MPC scheme and show that the original chance constraints remain satisfied at every time step. Furthermore, we show that…
University of Michigan（密歇根大学）；University of California, Riverside（加州大学河滨分校）
Real-world control applications often involve complex dynamics subject to abrupt changes or variations. Markov jump linear systems (MJS) provide a rich framework for modeling such dynamics. Despite an extensive history, theoretical understanding of parameter sensitivities of MJS control is somewhat lacking. Motivated by this, we investigate robustness aspects of certainty equivalent model-based optimal control for MJS with quadratic cost function. Given the uncertainty in the system matrices and in the Markov transition matrix is bounded by $ε$ and $η$ respectively, robustness results are esta…
Many probabilistic inference problems such as stochastic filtering or the computation of rare event probabilities require model analysis under initial and terminal constraints. We propose a solution to this bridging problem for the widely used class of population-structured Markov jump processes. The method is based on a state-space lumping scheme that aggregates states in a grid structure. The resulting approximate bridging distribution is used to iteratively refine relevant and truncate irrelevant parts of the state-space. This way the algorithm learns a well-justified finite-state projectio…
This paper presents a method for calculating the smoothed state distribution for Jump Markov Linear Systems. More specifically, the paper details a novel two-filter smoother that provides closed-form expressions for the smoothed hybrid state distribution. This distribution can be expressed as a Gaussian mixture with a known, but exponentially increasing, number of Gaussian components as the time index increases. This is accompanied by exponential growth in memory and computational requirements, which rapidly becomes intractable. To ameliorate this, we limit the number of allowed mixture terms …
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Markovian jump linear systems (MJLS) are an important class of dynamical systems that arise in many control applications. In this paper, we introduce the problem of controlling unknown (discrete-time) MJLS as a new benchmark for policy-based reinforcement learning of Markov decision processes (MDPs) with mixed continuous/discrete state variables. Compared with the traditional linear quadratic regulator (LQR), our proposed problem leads to a special hybrid MDP (with mixed continuous and discrete variables) and poses significant new challenges due to the appearance of an underlying Markov jump p…
