In this paper, we investigate the distributed convex optimization problem over a multi-agent system with Markovian switching communication networks. The objective function is the sum of each agent's local objective function, which cannot be known by other agents. The communication network is assumed to switch over a set of weight-balanced directed graphs with a Markovian property.We propose a consensus sub-gradient algorithm with two time-scale step-sizes to handle the uncertainty due to the Markovian switching topologies and the absence of global gradient information. With a proper selection of step-sizes, we prove the almost sure convergence of all agents' local estimates to the same optimal solution when the union graph of the Markovian network' states is strongly connected and the Markovian network is irreducible. Simulations are given for illustration of the results.
We study a finite-dimensional continuous-time optimal control problem on finite horizon for a controlled diffusion driven by Brownian motion, in the linear-quadratic case. We admit stochastic coefficients, possibly depending on an underlying independent marked point process, so that our model is general enough to include controlled switching systems where the switching mechanism is not required to be Markovian. The problem is solved by means of a Riccati equation, which a backward stochastic differential equation driven by the Bronwian motion and by the random measure associated to the marked …
The optimization of process economics within the model predictive control (MPC) formulation has given rise to a new control paradigm known as economic MPC (EMPC). Several authors have discussed the closed-loop properties of EMPC-controlled deterministic systems, however, little have uncertain systems been studied. In this paper we propose EMPC formulations for nonlinear Markovian switching systems which guarantee recursive feasibility, asymptotic performance bounds and constrained mean square (MS) stability.
In this paper we prove a weak necessary and sufficient maximum principle for Markovian regime switching stochastic optimal control problems. Instead of insisting on the maximum condition of the Hamiltonian, we show that 0 belongs to the sum of Clarke's generalized gradient of the Hamiltonian and Clarke's normal cone of the control constraint set at the optimal control. Under a joint concavity condition on the Hamiltonian and a convexity condition on the terminal objective function, the necessary condition becomes sufficient. We give four examples to demonstrate the weak stochastic maximum prin…
This work focuses on stability analysis of numerical solutions to jump diffusions and jump diffusions with Markovian switching. Due to the use of Poisson processes, using asymptotic expansions as in the usual approach of treating diffusion processes does not work. Different from the existing treatments of Euler-Maurayama methods for solutions of stochastic differential equations, we use techniques from stochastic approximation. We analyze the almost sure exponential stability and exponential $p$-stability. The benchmark test model in numerical solutions, namely, one-dimensional linear scalar j…
This paper first describes a class of uncertain stochastic control systems with Markovian switching, and derives an Itô-Liu formula for Markov-modulated processes. And we characterize an optimal control law, which satisfies the generalized Hamilton-Jacobi-Bellman (HJB) equation with Markovian switching. Then, by using the generalized HJB equation, we deduce the optimal consumption and portfolio policies under uncertain stochastic financial markets with Markovian switching. Finally, for constant relative risk-aversion (CRRA) felicity functions, we explicitly obtain the optimal consumption and p…
In this paper, we investigate the open-loop and weak closed-loop solvabilities of stochastic linear quadratic (LQ, for short) optimal control problem of Markovian regime switching system. Interestingly, these two solvabilities are equivalent. We first provide an alternative characterization of the open-loop solvability of the LQ problem using the perturbation approach. Then, we study the weak closed-loop solvability of the LQ problem of Markovian regime switching system, and establish the equivalent relationship between open-loop and weak closed-loop solvabilities. Finally, we present an examp…
This paper thoroughly investigates stochastic linear-quadratic optimal control problems with the Markovian regime switching system, where the coefficients of the state equation and the weighting matrices of the cost functional are random. We prove the solvability of the stochastic Riccati equation under the uniform convexity condition and obtain the closed-loop representation of the open-loop optimal control using the unique solvability of the corresponding stochastic Riccati equation. Moreover, by applying Itô's formula with jumps, we get a representation of the cost functional on a Hilbert s…
In this paper, we consider a continuous-time mean-variance portfolio selection with regime-switching and random horizon. Unlike previous works, the dynamic of assets are described by non-Markovian regime-switching models in the sense that all the market parameters are predictable with respect to the filtration generated jointly by Markov chain and Brownian motion. We formulate this problem as a constrained stochastic linear-quadratic optimal control problem. The Markov chain is assumed to be independent of the Brownian motion. So the market is incomplete. We derive closed-form expressions for …
We present a data-driven model predictive control scheme for chance-constrained Markovian switching systems with unknown switching probabilities. Using samples of the underlying Markov chain, ambiguity sets of transition probabilities are estimated which include the true conditional probability distributions with high probability. These sets are updated online and used to formulate a time-varying, risk-averse optimal control problem. We prove recursive feasibility of the resulting MPC scheme and show that the original chance constraints remain satisfied at every time step. Furthermore, we show…
In this paper, stability of linearly coupled dynamical systems with feedback pinning is studied. Event-triggered rules are employed on both diffusion coupling and feedback pinning to reduce the updating load of the coupled system. Here, both the coupling matrix and the set of pinned-nodes vary with time are induced by a homogeneous Markov chain. For each node, the diffusion coupling and feedback pinning are set up from the observation of its neighbors' and target's (if pinned) information at the latest event time and the next event time is triggered by some specified criteria. Two event-trigge…
This paper studies the consensusability problem of multi-agent systems (MASs), where agents communicate with each other through Markovian packet loss channels. We try to determine conditions under which there exists a linear distributed consensus controller such that the MAS can achieve mean square consensus. We first provide a necessary and sufficient consensus condition for MASs with single input and i.i.d.\ channel losses, which complements existing results. Then we proceed to study the case with identical Markovian packet losses. A necessary and sufficient consensus condition is firstly de…
Multidimensional indefinite stochastic Riccati equations and zero-sum stochastic linear-quadratic differential games with non-Markovian regime switching
In this paper, stability of linearly coupled dynamical systems with feedback pinning algorithm is studied. Here, both the coupling matrix and the set of pinned-nodes vary with time, induced by a continuous-time Markov chain with finite states. Event-triggered rules are employed on both diffusion coupling and feedback pinning terms, which can efficiently reduce the computation load, as well as communication load in some cases and be realized by the latest observations of the state information of its local neighborhood and the target trajectory. The next observation is triggered by certain crite…
In this paper, a distributed velocity-constrained consensus problem is studied for discrete-time multi-agent systems, where each agent's velocity is constrained to lie in a nonconvex set. A distributed constrained control algorithm is proposed to enable all agents to converge to a common point using only local information. {The gains of the algorithm for all agents need not to be the same or predesigned and can be adjusted by each agent itself based on its own and neighbors' information.} It is shown that the algorithm is robust to arbitrarily bounded communication delays and arbitrarily switc…
This paper introduces a new class of optimal switching problems, where the player is allowed to switch at a sequence of exogenous Poisson arrival times, and the underlying switching system is governed by an infinite horizon backward stochastic differential equation system. The value function and the optimal switching strategy are characterized by the solution of the underlying switching system. In a Markovian setting, the paper gives a complete description of the structure of switching regions by means of the comparison principle.
In this paper, we study the leader-following consensus problem for a class of uncertain nonlinear multi-agent systems under jointly connected directed switching networks. The uncertainty includes constant unbounded parameters and external disturbances. We first extend the recent result on the adaptive distributed observer from global asymptotical convergence to global exponential convergence. Then, by integrating the conventional adaptive control technique with the adaptive distributed observer, we present our solution by a distributed adaptive state feedback control law. Our result is illustr…
Consensus conditions and convergence speeds are crucial for distributed consensus algorithms of networked systems. Based on a basic first-order average-consensus protocol with time-varying topologies and additive noises, this paper first investigates its critical consensus condition on network topology by stochastic approximation frameworks. A new joint-connectivity condition called extensible joint-connectivity that contains a parameter $δ$ (termed the extensible exponent) is proposed. With this and a balanced topology condition, we show that a critical value of $δ$ for consensus is $1/2$. Op…
This paper studies the problem of the deterministic version of the Verification Theorem for the optimal m-states switching in infinite horizon under Markovian framework with arbitrary switching cost functions. The problem is formulated as an extended impulse control problem and solved by means of probabilistic tools such as the Snell envelop of processes and reflected backward stochastic differential equations. A viscosity solutions approach is employed to carry out a finne analysis on the associated system of m variational inequalities with inter-connected obstacles. We show that the vector o…
We solve non-Markovian optimal switching problems in discrete time on an infinite horizon, when the decision maker is risk aware and the filtration is general, and establish existence and uniqueness of solutions for the associated reflected backward stochastic difference equations. An example application to hydropower planning is provided.
Recently, the distributed state estimation problem for continuous-time linear systems over jointly connected switching networks was solved. It was shown that the estimation errors will asymptotically converge to the origin by using the generalized Barbalat's Lemma. This paper further studies the same problem with two new features. First, the asymptotic convergence is strengthened to the exponential convergence. This strengthened result not only offers a guaranteed convergence rate, but also renders the error system total stability and thus is able to withstand small disturbances. Second, the c…
In this paper, we study the $m$-states optimal switching problem in finite horizon, when the switching cost functions are arbitrary and can be positive or negative. This has an economic incentive in terms of central evaluation in cases where such organizations or state grants or financial assistance to power plants that promotes green energy in their production activity or what uses less polluting modes in their production. We show existence for optimal strategy via a verification theorem then we show existence and uniqueness of the value processes by using an approximation scheme. In the mark…
In this paper, we analyze the dynamics of spreading processes taking place over time-varying networks. A common approach to model time-varying networks is via Markovian random graph processes. This modeling approach presents the following limitation: Markovian random graphs can only replicate switching patterns with exponential inter-switching times, while in real applications these times are usually far from exponential. In this paper, we introduce a flexible and tractable extended family of processes able to replicate, with arbitrary accuracy, any distribution of inter-switching times. We th…
We study an open problem of risk-sensitive portfolio allocation in a regime-switching credit market with default contagion. The state space of the Markovian regime-switching process is assumed to be a countably infinite set. To characterize the value function, we investigate the corresponding recursive infinite-dimensional nonlinear dynamical programming equations (DPEs) based on default states. We propose to work in the following procedure: Applying the theory of monotone dynamical system, we first establish the existence and uniqueness of classical solutions to the recursive DPEs by a trunca…
This paper investigates the distributed event-based consensus problem of switching networks satisfying the jointly connected condition. Both the state consensus of homogeneous linear networks and output consensus of heterogeneous networks are studied. Two kinds of event-based protocols based on local sampled information are designed, without the need to solve any matrix equation or inequality. Theoretical analysis indicates that the proposed event-based protocols guarantee the achievement of consensus and the exclusion of Zeno behaviors for jointly connected undirected switching graphs. These …
This paper aims at proposing a procedure to derive distributed algorithms for distributed consensus-based optimization by using distributed algorithms for network resource allocation and vice versa over switching networks with/without synchronous protocol. It is shown that first-order gradient distributed consensus-based optimization algorithms can be used for finding an optimal solution of distributed resource allocation with synchronous protocol under weaker assumptions than those given in the literature for non-switching (static) networks. It is shown that first-order gradient distributed r…
Consensus and synchronization in discrete-time networks of multi-agents with stochastically switching topologies and time delays
Fudan University（复旦大学）；Max Planck Institute for Mathematics in the Sciences（马克斯·普朗克科学促进部数理研究所）；Santa Fe Institute for the Sciences of Complexity（圣塔菲复杂性科学研究所）
