This paper studies a discrete-time stochastic control problem with linear quadratic criteria over an infinite-time horizon. We focus on a class of control systems whose system matrices are associated with random parameters involving unknown statistical properties. In particular, we design a distributed Q-learning algorithm to tackle the Riccati equation and derive the optimal controller stabilizing the system. The key technique is that we convert the problem of solving the Riccati equation into deriving the zero point of a matrix equation and devise a distributed stochastic approximation method to compute the estimates of the zero point. The convergence analysis proves that the distributed Q-learning algorithm converges to the correct value eventually. A numerical example sheds light on that the distributed Q-learning algorithm converges asymptotically.
Distributed Output Regulation for a Class of Nonlinear Multi-Agent Systems with Unknown-Input Leaders
School of Automation, Beijing University of Posts and Telecommunications（北京邮电大学自动化学院）；Key Laboratory of Systems and Control, Institute of Systems Science, Chinese Academy of Sciences（中国科学院系统与控制重点实验室，系统科学研究所）
In this paper, we present a distributed algorithm for solving convex, constraint-coupled, optimization problems over peer-to-peer networks. We consider a network of processors that aim to cooperatively minimize the sum of local cost functions, subject to individual constraints and to global coupling constraints. The major assumption of this work is that the cost functions are unknown and must be learned online. We propose a fully distributed algorithm, based on a primal decomposition approach, that uses iteratively refined data-driven estimations of the cost functions over the iterations. The …
We investigate the problem of stabilizing an unknown networked linear system under communication constraints and adversarial disturbances. We propose the first provably stabilizing algorithm for the problem. The algorithm uses a distributed version of nested convex body chasing to maintain a consistent estimate of the network dynamics and applies system level synthesis to determine a distributed controller based on this estimated model. Our approach avoids the need for system identification and accommodates a broad class of communication delay while being fully distributed and scaling favorabl…
In this paper, we propose a chance constrained stochastic model predictive control scheme for reference tracking of distributed linear time-invariant systems with additive stochastic uncertainty. The chance constraints are reformulated analytically based on mean-variance information, where we design suitable Probabilistic Reachable Sets for constraint tightening. Furthermore, the chance constraints are proven to be satisfied in closed-loop operation. The design of an invariant set for tracking complements the controller and ensures convergence to arbitrary admissible reference points, while a …
Online optimization has recently opened avenues to study optimal control for time-varying cost functions that are unknown in advance. Inspired by this line of research, we study the distributed online linear quadratic regulator (LQR) problem for linear time-invariant (LTI) systems with unknown dynamics. Consider a multi-agent network where each agent is modeled as a LTI system. The network has a global time-varying quadratic cost, which may evolve adversarially and is only partially observed by each agent sequentially. The goal of the network is to collectively (i) estimate the unknown dynamic…
We present a distributed optimization algorithm for solving online personalized optimization problems over a network of computing and communicating nodes, each of which linked to a specific user. The local objective functions are assumed to have a composite structure and to consist of a known time-varying (engineering) part and an unknown (user-specific) part. Regarding the unknown part, it is assumed to have a known parametric (e.g., quadratic) structure a priori, whose parameters are to be learned along with the evolution of the algorithm. The algorithm is composed of two intertwined compone…
State estimation in water distribution networks (WDN), the problem of estimating all unknown network heads and flows given select measurements, is challenging due to the nonconvexity of hydraulic models and significant uncertainty from water demands, network parameters, and measurements. To this end, a probabilistic modeling for state estimation (PSE) in WDNs is proposed. After linearizing the nonlinear hydraulic WDN model, the proposed PSE shows that the covariance matrix of unknown system states (unmeasured heads and flows) can be linearly expressed by the covariance matrix of three uncertai…
We study the problem of distributed Kalman filtering for sensor networks in the presence of model uncertainty. More precisely, we assume that the actual state-space model belongs to a ball, in the Kullback-Leibler topology, about the nominal state-space model and whose radius reflects the mismatch modeling budget allowed for each time step. We propose a distributed Kalman filter with diffusion step which is robust with respect to the aforementioned model uncertainty. Moreover, we derive the corresponding least favorable performance. Finally, we check the effectiveness of the proposed algorithm…
In this paper, we consider a Nash equilibrium seeking problem for a class of high-order multi-agent systems with unknown dynamics. Different from existing results for single integrators, we aim to steer the outputs of this class of uncertain high-order agents to the Nash equilibrium of some noncooperative game in a distributed manner. To overcome the difficulties brought by the high-order structure, unknown nonlinearities, and the regulation requirement, we first introduce a virtual player for each agent and solve an auxiliary noncooperative game for them. Then, we develop a distributed adapti…
This paper studies an optimal consensus problem for a group of heterogeneous high-order agents with unknown control directions. Compared with existing consensus results, the consensus point is further required to an optimal solution to some distributed optimization problem. To solve this problem, we first augment each agent with an optimal signal generator to reproduce the global optimal point of the given distributed optimization problem, and then complete the global optimal consensus design by developing some adaptive tracking controllers for these augmented agents. Moreover, we present an e…
In stochastic Nash equilibrium problems (SNEPs), it is natural for players to be uncertain about their complex environments and have multi-dimensional unknown parameters in their models. Among various SNEPs, this paper focuses on locally coupled network games where the objective of each rational player is subject to the aggregate influence of its neighbors. We propose a distributed learning algorithm based on the proximal-point iteration and ordinary least-square estimator, where each player repeatedly updates the local estimates of neighboring decisions, makes its augmented best-response deci…
Adaptive cooperative tracking control with prescribed performance function (PPF) is proposed for high-order nonlinear multi-agent systems. The tracking error originally within a known large set is confined to a smaller predefined set using this approach. Using output error transformation, the constrained system is relaxed and mapped to an unconstrained one. The controller is conceived under the assumption that the agents' nonlinear dynamics are unknown and the perceived network is structured and strongly connected. Under the proposed controller, all agents track the trajectory of the leader no…
This paper focuses on the consensus and formation problems of multiagent systems under unknown persistent disturbances. Specifically, we propose a novel method that combines an existing consensus (or formation) algorithm with a new controller. The new controller has an integral action that produces a control input based on an error signal locally projected onto the column space of the graph Laplacian. This action allows agents to achieve a consensus or a predetermined formation objective under constant or time-varying disturbances. This study has analyzed the stability properties of this archi…
Distributed Nash equilibrium seeking for games in uncertain networked systems without a prior knowledge about control directions is explored in this paper. More specifically, the dynamics of the players are supposed to be first-order or second-order systems in which the control directions are unknown and there are parametric uncertainties. To achieve Nash equilibrium seeking in a distributed way, Nussbaum function based strategies are proposed through separately designing an optimization module and a state regulation module. The optimization module generates a reference trajectory, that can se…
This paper presents a Distributed Stochastic Model Predictive Control algorithm for networks of linear systems with multiplicative uncertainties and local chance constraints on the states and control inputs. The chance constraints are approximated via Cantelli's inequality by means of expected value and covariance. The cooperative control algorithm is based on the distributed Alternating Direction Method of Multipliers, which renders the controller fully distributedly implementable, recursively feasible and ensures point-wise convergence of the states. The aforementioned properties are guarant…
In this note, the problem of simultaneous leader-following consensus and parameter estimation is studied for a class of multi-agent systems subject to an uncertain leader system. The leader system is described by a sum of sinusoids with unknown amplitudes, frequencies and phases. A distributed adaptive observer is established for each agent to estimate the unknown frequencies of the leader. It is shown that if the signal of the leader is sufficiently rich, the estimation errors of the unknown frequencies converge to zero asymptotically for all the agents. Based on the designed distributed adap…
A distributed nonsmooth robust resource allocation problem with cardinality constrained uncertainty is investigated in this paper. The global objective is consisted of local objectives, which are convex but nonsmooth. Each agent is constrained in its private convex set and has only the information of its corresponding local objective. The resource allocation condition is subject to the cardinality constrained uncertainty sets. By employing the duality theory of convex optimization, a dual problem of the robust resource allocation problem is presented. For solving this dual problem, a distribut…
We present AUQ-ADMM, an adaptive uncertainty-weighted consensus ADMM method for solving large-scale convex optimization problems in a distributed manner. Our key contribution is a novel adaptive weighting scheme that empirically increases the progress made by consensus ADMM scheme and is attractive when using a large number of subproblems. The weights are related to the uncertainty associated with the solutions of each subproblem, and are efficiently computed using low-rank approximations. We show AUQ-ADMM provably converges and demonstrate its effectiveness on a series of machine learning app…
A design framework recently has been developed to stabilize interconnected multiagent systems in a distributed manner, and systematically capture the architectural aspect of cyber-physical systems. Such a control theoretic framework, however, results in a stabilization protocol which is passive with respect to the cyber attacks and conservative regarding the guaranteed level of resiliency. We treat the control layer topology and stabilization gains as the degrees of freedom, and develop a mixed control and cybersecurity design framework to address the above concerns. From a control perspective…
We consider a discrete time stochastic Markovian control problem under model uncertainty. Such uncertainty not only comes from the fact that the true probability law of the underlying stochastic process is unknown, but the parametric family of probability distributions which the true law belongs to is also unknown. We propose a nonparametric adaptive robust control methodology to deal with such problem. Our approach hinges on the following building concepts: first, using the adaptive robust paradigm to incorporate online learning and uncertainty reduction into the robust control problem; secon…
University at Buffalo（纽约州立大学布法罗分校）；University of Michigan, Ann Arbor（密歇根大学安娜堡分校）
Fair resource allocation is one of the most important topics in communication networks. Existing solutions almost exclusively assume each user utility function is known and concave. This paper seeks to answer the following question: how to allocate resources when utility functions are unknown, even to the users? This answer has become increasingly important in the next-generation AI-aware communication networks where the user utilities are complex and their closed-forms are hard to obtain. In this paper, we provide a new solution using a distributed and data-driven bilevel optimization approac…
In this paper, we present Lyapunov-based adaptive controllers for the practical (or real) stabilization of a perturbed chain of integrators with bounded uncertainties. We refer to such controllers as Adaptive Higher Order Sliding Mode (AHOSM) controllers since they are designed for nonlinear SISO systems with bounded uncertainties such that the uncertainty bounds are unknown. Our main result states that, given any neighborhood N of the origin, we determine a controller insuring, for every uncertainty bounds, that every trajectory of the corresponding closed loop system enters N and eventually …
Many engineering systems are subject to spatially distributed uncertainty, i.e. uncertainty that can be modeled as a random field. Altering the mean or covariance of this uncertainty will in general change the statistical distribution of the system outputs. We present an approach for computing the sensitivity of the statistics of system outputs with respect to the parameters describing the mean and covariance of the distributed uncertainty. This sensitivity information is then incorporated into a gradient-based optimizer to optimize the structure of the distributed uncertainty to achieve desir…
Motivated by broad applications in various fields of engineering, we study a network resource allocation problem where the goal is to optimally allocate a fixed quantity of resources over a network of nodes. We consider large scale networks with complex interconnection structures, thus any solution must be implemented in parallel and based only on local data resulting in a need for distributed algorithms. In this paper, we study a distributed Lagrangian method for such problems. By utilizing the so-called distributed subgradient methods to solve the dual problem, our approach eliminates the ne…
This paper studies resilient distributed estimation under measurement attacks. A set of agents each makes successive local, linear, noisy measurements of an unknown vector field collected in a vector parameter. The local measurement models are heterogeneous across agents and may be locally unobservable for the unknown parameter. An adversary compromises some of the measurement streams and changes their values arbitrarily. The agents' goal is to cooperate over a peer-to-peer communication network to process their (possibly compromised) local measurements and estimate the value of the unknown ve…
This paper considers a class of mean field linear-quadratic-Gaussian (LQG) games with model uncertainty. The drift term in the dynamics of the agents contains a common unknown function. We take a robust optimization approach where a representative agent in the limiting model views the drift uncertainty as an adversarial player. By including the mean field dynamics in an augmented state space, we solve two optimal control problems sequentially, which combined with consistent mean field approximations provides a solution to the robust game. A set of decentralized control strategies is derived by…
We consider a Markov decision process subject to model uncertainty in a Bayesian framework, where we assume that the state process is observed but its law is unknown to the observer. In addition, while the state process and the controls are observed at time $t$, the actual cost that may depend on the unknown parameter is not known at time $t$. The controller optimizes total cost by using a family of special risk measures, that we call risk filters and that are appropriately defined to take into account the model uncertainty of the controlled system. These key features lead to non-standard and …
We study the distribution and uncertainty of nonconvex optimization for noisy tensor completion -- the problem of estimating a low-rank tensor given incomplete and corrupted observations of its entries. Focusing on a two-stage estimation algorithm proposed by Cai et al. (2019), we characterize the distribution of this nonconvex estimator down to fine scales. This distributional theory in turn allows one to construct valid and short confidence intervals for both the unseen tensor entries and the unknown tensor factors. The proposed inferential procedure enjoys several important features: (1) it…
