Efforts in this paper seek to combine graph theory with adaptive dynamic programming (ADP) as a reinforcement learning (RL) framework to determine forward-in-time, real-time, approximate optimal controllers for distributed multi-agent systems with uncertain nonlinear dynamics. A decentralized continuous time-varying control strategy is proposed, using only local communication feedback from two-hop neighbors on a communication topology that has a spanning tree. An actor-critic-identifier architecture is proposed that employs a nonlinear state derivative estimator to estimate the unknown dynamics online and uses the estimate thus obtained for value function approximation.
In smart grids, distributed energy resources (DERs) have penetrated residential zones to provide a new form of electricity supply, mainly from renewable energy. Residential households and commercial buildings with DERs have become prosumers in the local grids, since they can sell surplus power to others. Researches have been initiated to integrate and utilize DERs through better control and communication strategies. With the advances in the Internet of Things (IoT) technology, unprecedented coordination among DERs can be achieved to facilitate energy trading and transactive energy management. …
We study the decentralized consensus and stochastic optimization problems with compressed communications over static directed graphs. We propose an iterative gradient-based algorithm that compresses messages according to a desired compression ratio. The proposed method provably reduces the communication overhead on the network at every communication round. Contrary to existing literature, we allow for arbitrary compression ratios in the communicated messages. We show a linear convergence rate for the proposed method on the consensus problem. Moreover, we provide explicit convergence rates for …
In decentralized optimization, multiple nodes in a network collaborate to minimize the sum of their local loss functions. The information exchange between nodes required for this task, is often limited by network connectivity. We consider a setting in which communication between nodes is hindered by both (i) a finite rate-constraint on the signal transmitted by any node, and (ii) additive noise corrupting the signal received by any node. We propose a novel algorithm for this scenario: Decentralized Lazy Mirror Descent with Differential Exchanges (DLMD-DiffEx), which guarantees convergence of t…
Motivated by decentralized sensing and policy evaluation problems, we consider a particular type of distributed stochastic optimization problem over a network, called the online stochastic distributed averaging problem. We design a dual-based method for this distributed consensus problem with Polyak--Ruppert averaging and analyze its behavior. We show that the proposed algorithm attains an accelerated deterministic error depending optimally on the condition number of the network, and also that it has an order-optimal stochastic error. This improves on the guarantees of state-of-the-art distrib…
Power packets are proposed as a transmission unit that can deliver power and information simultaneously. They are transferred using the store-and-forward method of power routers. A system that achieves power supply/demand in this manner is called a power packet network (PPN). A PPN is expected to enhance structural robustness and operational reliability in an energy storage system (ESS) with recent diverse distributed sources. However, this technology is still in its early stage, and faces numerous challenges, such as high cost of implementation and complicated energy management. In this paper…
With the advancement in smart grid and smart energy devices, demand response becomes one of the most economic and feasible solutions to ease the load stress of the power grids during peak hours. In this work, we propose a fully decentralized ensemble control framework with consensus for demand response (DR) events and compatible control methods based on random policies. We show that under the consensus that is tailored to DR, our proposed decentralized control method yields the same optimality as the centralized control method in both myopic and multistage settings.
Decentralized network theories focus on achieving consensus and in speeding up the rate of convergence to consensus. However, network cohesion (i.e., maintaining consensus) during transitions between consensus values is also important when transporting flexible structures. Deviations in the robot positions due to loss of cohesion when moving flexible structures from one position to another, such as uncuredcomposite aircraft wings, can cause large deformations, which in turn, can result in potential damage. The major contribution of this work is to develop a decentralized approach to transport …
Texas A&M University（德克萨斯农工大学）；University of Minnesota（明尼苏达大学）
We consider a distributed non-convex optimization where a network of agents aims at minimizing a global function over the Stiefel manifold. The global function is represented as a finite sum of smooth local functions, where each local function is associated with one agent and agents communicate with each other over an undirected connected graph. The problem is non-convex as local functions are possibly non-convex (but smooth) and the Steifel manifold is a non-convex set. We present a decentralized Riemannian stochastic gradient method (DRSGD) with the convergence rate of $\mathcal{O}(1/\sqrt{K…
John A. Paulson School of Engineering and Applied Sciences, Harvard University（哈佛大学约翰·A·保尔森工程与应用科学学院）
This paper considers a distributed reinforcement learning problem for decentralized linear quadratic control with partial state observations and local costs. We propose a Zero-Order Distributed Policy Optimization algorithm (ZODPO) that learns linear local controllers in a distributed fashion, leveraging the ideas of policy gradient, zero-order optimization and consensus algorithms. In ZODPO, each agent estimates the global cost by consensus, and then conducts local policy gradient in parallel based on zero-order gradient estimation. ZODPO only requires limited communication and storage even i…
Towards Integrated Traffic Control with Operating Decentralized Autonomous Organization
Shanghai AI Lab（上海人工智能实验室）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；RIOH High and Technology Group（国研翰高科技集团）；Macau Institute of Systems Engineering（澳门系统工程研究所）
Towards Serverless Semi-Decentralized Federated Learning with Heterogeneous Optimizers
Department of Electrical and Computer Engineering, Princeton University（普林斯顿大学电子工程与计算机科学系）；Department of Electrical and Computer Engineering, Purdue University（帕克森大学电子工程与计算机科学系）
In this work, we propose a novel safe and scalable decentralized solution for multi-agent control in the presence of stochastic disturbances. Safety is mathematically encoded using stochastic control barrier functions and safe controls are computed by solving quadratic programs. Decentralization is achieved by augmenting to each agent's optimization variables, copy variables, for its neighbors. This allows us to decouple the centralized multi-agent optimization problem. However, to ensure safety, neighboring agents must agree on "what is safe for both of us" and this creates a need for consens…
Distributed consensus mechanisms have been widely researched and made popular with a number of blockchain-based token applications, such as Bitcoin, and Ethereum. Although these general-purpose platforms have matured for scale and security, they are designed for human incentive and continue to require currency reward and contract functions that are not requisite in machine communications. Redes Chain is a custom designed blockchain, built to support fully decentralized self-organization in wireless networks-without a cryptocurrency or contract dependency.
