This paper studies distributed Q-learning for Linear Quadratic Regulator (LQR) in a multi-agent network. The existing results often assume that agents can observe the global system state, which may be infeasible in large-scale systems due to privacy concerns or communication constraints. In this work, we consider a setting with unknown system models and no centralized coordinator. We devise a state tracking (ST) based Q-learning algorithm to design optimal controllers for agents. Specifically, we assume that agents maintain local estimates of the global state based on their local information and communications with neighbors. At each step, every agent updates its local global state estimation, based on which it solves an approximate Q-factor locally through policy iteration. Assuming decaying injected excitation noise during the policy evaluation, we prove that the local estimation converges to the true global state, and establish the convergence of the proposed distributed ST-based Q-learning algorithm. The experimental studies corroborate our theoretical results by showing that our proposed method achieves comparable performance with the centralized case.
Distributed Dynamic Map Fusion via Federated Learning for Intelligent Networked Vehicles
Southern University of Science and Technology（南方科技大学）；Sifakis Research Institute of Trustworthy Autonomous Systems（思法基可信自主系统研究院）；Pazhou Lab（琶洲实验室）；Harbin Institute of Technology（哈尔滨工业大学）
Distributed Zeroth-Order Policy Gradient for Networked Multi-agent Reinforcement Learning from Human Feedback
Singapore University of Technology and Design（新加坡科技设计大学）；Southeast University（东南大学）；University of California, Riverside（加州大学河滨分校）；Sichuan University（四川大学）；Purple Mountain Laboratories（紫金山实验室）
COIN-GP: Cooperative Online Learning in Networked Distributed Systems with Partial Measurements via Gaussian Process Regression
Technical University of Munich（慕尼黑工业大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Tongji University（同济大学）；Shanghai Yangzhi Rehabilitation Hospital（上海市养志康复医院）；University of Science and Technology of China（中国科学技术大学）；The Hong Kong Polytechnic University（香港理工大学）
Distributed scalable coupled policy algorithm for networked multi-agent reinforcement learning
Singapore University of Technology and Design（新加坡科技设计大学）；University of California, Riverside（加州大学河滨分校）；Southeast University（东南大学）；Purple Mountain Laboratories（紫金山实验室）
