This paper provides a compositional scheme based on dissipativity approaches for constructing finite abstractions of continuous-time continuous-space stochastic control systems. The proposed framework enjoys the structure of the interconnection topology and employs a notion of stochastic storage functions, that describe joint dissipativity-type properties of subsystems and their abstractions. By utilizing those stochastic storage functions, one can establish a relation between continuous-time continuous-space stochastic systems and their finite counterparts while quantifying probabilistic distances between their output trajectories. Consequently, one can employ the finite system as a suitable substitution of the continuous-time one in the controller design process with a guaranteed error bound. In this respect, we first leverage dissipativity-type compositional conditions for the compositional quantification of the distance between the interconnection of continuous-time continuous-space stochastic systems and that of their discrete-time (finite or infinite) abstractions. We then consider a specific class of stochastic affine systems and construct their finite abstractions together with their corresponding stochastic storage functions. The effectiveness of the proposed results is demonstrated by applying them to a temperature regulation in a circular network containing 100 rooms and compositionally constructing a discrete-time abstraction from its original continuous-time dynamic. The constructed discrete-time abstraction is then utilized as a substitute to compositionally synthesize policies keeping the temperature of each room in a comfort zone.
This paper discusses the functional stability of closed-loop Markov Chains under optimal policies resulting from a discounted optimality criterion, forming Markov Decision Processes (MDPs). We investigate the stability of MDPs in the sense of probability measures (densities) underlying the state distributions and extend the dissipativity theory of Economic Model Predictive Control in order to characterize the MDP stability. This theory requires a so-called storage function satisfying a dissipativity inequality. In the probability measures space and for the discounted setting, we introduce new …
In this paper, we propose a compositional approach for the construction of finite abstractions (a.k.a. finite Markov decision processes (MDPs)) for networks of discrete-time stochastic control subsystems that are not necessarily stabilizable. The proposed approach leverages the interconnection topology and a notion of finite-step stochastic storage functions, that describes joint dissipativity-type properties of subsystems and their abstractions, and establishes a finite-step stochastic simulation function as a relation between the network and its abstraction. To this end, we first develop a n…
We propose a compositional approach for constructing abstractions of general Markov decision processes using approximate probabilistic relations. The abstraction framework is based on the notion of $δ$-lifted relations, using which one can quantify the distance in probability between the interconnected gMDPs and that of their abstractions. This new approximate relation unifies compositionality results in the literature by incorporating the dependencies between state transitions explicitly and by allowing abstract models to have either finite or infinite state spaces. Accordingly, one can lever…
In this work, we propose a compositional scheme for the safety controller synthesis of interconnected discrete-time stochastic systems with Markovian switching signals. Our proposed approach is based on a notion of so-called control storage certificates computed for individual subsystems, by leveraging which, one can synthesize state-feedback controllers for interconnected systems to enforce safety specifications over finite time horizons. To do so, we employ a sum-of-squares (SOS) optimization approach to search for multiple storage certificates of each switching subsystem while synthesizing …
In this paper, we provide a compositional approach for constructing finite abstractions (a.k.a. finite Markov decision processes (MDPs)) of interconnected discrete-time stochastic switched systems. The proposed framework is based on a notion of stochastic simulation functions, using which one can employ an abstract system as a substitution of the original one in the controller design process with guaranteed error bounds on their output trajectories. To this end, we first provide probabilistic closeness guarantees between the interconnection of stochastic switched subsystems and that of their f…
Compositional Construction of Infinite Abstractions for Networks of Stochastic Control Systems
Technical University of Munich（慕尼黑工业大学）；Newcastle University（纽卡斯尔大学）；University of Colorado Boulder（科罗拉多大学博尔德分校）；Ludwig Maximilian University of Munich（慕尼黑大学）
This paper is concerned with a compositional scheme for the construction of control barrier certificates for interconnected discrete-time stochastic systems. The main objective is to synthesize switching control policies against $ω$-regular properties that can be described by accepting languages of deterministic Streett automata (DSA) along with providing probabilistic guarantees for the satisfaction of such specifications. The proposed framework leverages the interconnection topology and a notion of so-called \emph{control sub-barrier certificates} of subsystems, which are used to composition…
