Degrees of Freedom in Multi-Agent Coordination: Between Centralized and Decentralized Regimes
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Wake Forest University（维克森林大学）；Shandong University（山东大学）；University of Science and Technology of China（中国科学技术大学）
Large language model (LLM)-based multi-agent systems (MAS) are commonly organized around two coordination regimes. Centralized coordination provides explicit control but can create a bottleneck at a single coordination locus, whereas decentralized coordination offers local autonomy but must achieve effective collaboration through distributed decisions and experience. These trade-offs motivate a systematic study of the space between the two regimes. We introduce Degrees of Freedom (DoF), a continuous configuration framework that quantifies the size of the decision space delegated to downstream agents. DoF controls the granularity of delegated work and the target probability that each task unit is assigned to the branches. Its endpoints recover the two coordination regimes. Across multiple task domains, DoF reveals a hierarchy of organizational effects. Delegated granularity and within-regime routing produce structured but bounded profile reallocations, whereas task domain and retained experience are associated with stronger differences between the centralized- and decentralized-branch profiles. The resulting performance landscape is task-dependent: different domains favor different DoF configurations, while atomic-target conditions are more consistent across runs. In-domain retained experience brings the two branch profiles closer together. The same controlled sweep reveals a task-dependent coordination landscape: some domains exhibit higher-performing configurations, whereas others show flatter responses and no single configuration dominates across domains. In five of the six evaluated state, an intermediate DoF configuration achieves higher mean accuracy than both endpoints, demonstrating the value of exploring the coordination space between them. Together, these findings provide empirical guidance for studying and designing intermediate coordination regimes in LLM-based MAS.
Centralized Training for Decentralized Execution, where agents are trained offline using centralized information but execute in a decentralized manner online, has gained popularity in the multi-agent reinforcement learning community. In particular, actor-critic methods with a centralized critic and decentralized actors are a common instance of this idea. However, the implications of using a centralized critic in this context are not fully discussed and understood even though it is the standard choice of many algorithms. We therefore formally analyze centralized and decentralized critic approac…
Is Centralized Training with Decentralized Execution Framework Centralized Enough for MARL?
Zhejiang University（浙江大学）；Zhejiang Provincial Engineering Research Center for Real-Time SmartTech in Urban Security Governance, School of Computer and Computing Science, Hangzhou City University（浙江省实时智慧城市治理实时智能技术工程研究中心，计算机与计算科学学院，杭州市大学）；State Key Laboratory of Blockchain and Data Security, Zhejiang University（区块链与数据安全国家重点实验室，浙江大学）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新技术区（滨江区）区块链与数据安全研究院）
Stanford University（斯坦福大学）；Microsoft（微软）；MIT Lincoln Labs（麻省理工学院林肯实验室）
Multi-agent reinforcement learning (MARL) requires coordination to efficiently solve certain tasks. Fully centralized control is often infeasible in such domains due to the size of joint action spaces. Coordination graph based formalization allows reasoning about the joint action based on the structure of interactions. However, they often require domain expertise in their design. This paper introduces the deep implicit coordination graph (DICG) architecture for such scenarios. DICG consists of a module for inferring the dynamic coordination graph structure which is then used by a graph neural …
Decentralized Transformers with Centralized Aggregation are Sample-Efficient Multi-Agent World Models
Tsinghua University（清华大学）；Institute of Artificial Intelligence (TeleAI), China Telecom（人工智能研究所（TeleAI），中国电信）；Shanghai AI Laboratory（上海人工智能实验室）；Shanghai Jiaotong University（上海交通大学）
Khoury College of Computer Sciences, Northeastern University（东北大学库里计算机科学学院）
In many real-world multi-robot tasks, high-quality solutions often require a team of robots to perform asynchronous actions under decentralized control. Decentralized multi-agent reinforcement learning methods have difficulty learning decentralized policies because of the environment appearing to be non-stationary due to other agents also learning at the same time. In this paper, we address this challenge by proposing a macro-action-based decentralized multi-agent double deep recurrent Q-net (MacDec-MADDRQN) which trains each decentralized Q-net using a centralized Q-net for action selection. …
Shanghai Jiao Tong University（上海交通大学）；Tencent AI Lab（腾讯人工智能实验室）；University College London（伦敦大学学院）；Carnegie Mellon University（卡内基梅隆大学）；Huya AI（虎牙人工智能）
In many real-world problems, a team of agents need to collaborate to maximize the common reward. Although existing works formulate this problem into a centralized learning with decentralized execution framework, which avoids the non-stationary problem in training, their decentralized execution paradigm limits the agents' capability to coordinate. Inspired by the concept of correlated equilibrium, we propose to introduce a coordination signal to address this limitation, and theoretically show that following mild conditions, decentralized agents with the coordination signal can coordinate their …
IMT Lille Douai（里尔-杜埃国立高等矿业电信学院）；Université de Lille（里尔大学）；University Rey Juan Carlos（胡安卡洛斯国王大学）；Valencian Research Institute for AI（瓦伦西亚人工智能研究院）；Universitat Politècnica de València（瓦伦西亚理工大学）
Open and shared manufacturing factories typically dispose of a limited number of robots that should be properly allocated to tasks in time and space for an effective and efficient system performance. In particular, we deal with the dynamic capacitated production planning problem with sequence independent setup costs where quantities of products to manufacture and location of robots need to be determined at consecutive periods within a given time horizon and products can be anticipated or backordered related to the demand period. We consider a decentralized multi-agent variant of this problem i…
Peking University（北京大学）；Tencent AI Lab（腾讯人工智能实验室）；BAAI（北京智源人工智能研究院）
In cooperative multi-agent reinforcement learning (MARL), combining value decomposition with actor-critic enables agents to learn stochastic policies, which are more suitable for the partially observable environment. Given the goal of learning local policies that enable decentralized execution, agents are commonly assumed to be independent of each other, even in centralized training. However, such an assumption may prohibit agents from learning the optimal joint policy. To address this problem, we explicitly take the dependency among agents into centralized training. Although this leads to the…
University of Science and Technology of China（中国科学技术大学）；Huawei Cloud（华为云）
Due to the partial observability and communication constraints in many multi-agent reinforcement learning (MARL) tasks, centralized training with decentralized execution (CTDE) has become one of the most widely used MARL paradigms. In CTDE, centralized information is dedicated to learning the allocation of the team reward with a mixing network, while the learning of individual Q-values is usually based on local observations. The insufficient utility of global observation will degrade performance in challenging environments. To this end, this work proposes a novel Centralized Teacher with Decen…
University of California Berkeley（加州大学伯克利分校）；Caltech（加州理工学院）
We study the problem of online learning in competitive settings in the context of two-sided matching markets. In particular, one side of the market, the agents, must learn about their preferences over the other side, the firms, through repeated interaction while competing with other agents for successful matches. We propose a class of decentralized, communication- and coordination-free algorithms that agents can use to reach to their stable match in structured matching markets. In contrast to prior works, the proposed algorithms make decisions based solely on an agent's own history of play and…
University of Delaware（特拉华大学）；U.S. Army Research Laboratory（美国陆军研究实验室）
Recent multi-agent actor-critic methods have utilized centralized training with decentralized execution to address the non-stationarity of co-adapting agents. This training paradigm constrains learning to the centralized phase such that only pre-learned policies may be used during the decentralized phase, which performs poorly when agent communications are delayed, noisy, or disrupted. In this work, we propose a new system that can gracefully handle partially-observable information due to communication disruptions during decentralized execution. Our approach augments the multi-agent actor-crit…
Multi-agent In-context Coordination via Decentralized Memory Retrieval
National Key Laboratory of Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；School of Artificial Intelligence, Nanjing University（南京大学人工智能学院）；Tencent（腾讯）
Resilient Charging Infrastructure via Decentralized Coordination of Electric Vehicles at Scale
School of Computer Science, University of Leeds, UK（计算机科学学院，利兹大学）；AI-MAS Laboratory, National University of Science and Technology POLITEHNICA Bucharest（AI-MAS实验室，波兰技术大学布拉科夫分校）
École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）
Rapid advances in data collection and processing capabilities have allowed for the use of increasingly complex models that give rise to nonconvex optimization problems. These formulations, however, can be arbitrarily difficult to solve in general, in the sense that even simply verifying that a given point is a local minimum can be NP-hard [1]. Still, some relatively simple algorithms have been shown to lead to surprisingly good empirical results in many contexts of interest. Perhaps the most prominent example is the success of the backpropagation algorithm for training neural networks. Several…
Load aggregators are independent private entities whose goal is to optimize energy consumption flexibilities offered by multiple residential consumers. Although aggregators optimize their decisions in a decentralized way, they are indirectly linked together if their respective consumers belong to the same distribution grid. This is an important issue for a distribution system operator (DSO), in charge of the reliability of the distribution network, it has to ensure that decentralized decisions taken do not violate the grid constraints and do not increase the global system costs. From the infor…
Public and private blockchain for decentralized digital building twins and building automation system
School of Technology, Eastern Illinois University（东部伊利诺伊大学技术学院）；Myers-Lawson School of Construction, Virginia Polytechnic Institute and State University（弗吉尼亚理工学院和州立大学梅斯-劳森建设学院）
Robotic systems, working together as a team, are becoming valuable players in different real-world applications, from disaster response to warehouse fulfillment services. Centralized solutions for coordinating multi-robot teams often suffer from poor scalability and vulnerability to communication disruptions. This paper develops a decentralized multi-agent task allocation (Dec-MATA) algorithm for multi-robot applications. The task planning problem is posed as a maximum-weighted matching of a bipartite graph, the solution of which using the blossom algorithm allows each robot to autonomously id…
While various multi-agent reinforcement learning methods have been proposed in cooperative settings, few works investigate how self-interested learning agents achieve mutual coordination in decentralized general-sum games and generalize pre-trained policies to non-cooperative opponents during execution. In this paper, we present Generalizable Risk-Sensitive Policy (GRSP). GRSP learns the distributions over agent's return and estimate a dynamic risk-seeking bonus to discover risky coordination strategies. Furthermore, to avoid overfitting to training opponents, GRSP learns an auxiliary opponent…
Multi-agent reinforcement learning (MARL) is a promising framework for solving complex tasks with many agents. However, a key challenge in MARL is defining private utility functions that ensure coordination when training decentralized agents. This challenge is especially prevalent in unstructured tasks with sparse rewards and many agents. We show that successor features can help address this challenge by disentangling an individual agent's impact on the global value function from that of all other agents. We use this disentanglement to compactly represent private utilities that support stable …
The infectious diseases are spreading due to human interactions enabled by various social networks. Therefore, when a new pathogen such as SARS-CoV-2 causes an outbreak, the non-pharmaceutical isolation strategies (e.g., social distancing) are the only possible response to disrupt its spreading. To this end, we introduce the new epidemic model (SICARS) and compare the centralized (C), decentralized (D), and combined (C+D) social distancing strategies, and analyze their efficiency to control the dynamics of COVID-19 on heterogeneous complex networks. Our analysis shows that the centralized soci…
Collective Privacy Recovery: Data-sharing Coordination via Decentralized Artificial Intelligence
School of Computing, University of Leeds（利兹大学计算学院）；ETH Zurich（苏黎世联邦理工学院）；Institute for a Secure and Sustainable Environment, University of Tennessee（田纳西大学安全与可持续环境研究所）；University of Tennessee（田纳西大学）
To achieve an optimal outcome in many situations, agents need to choose distinct actions from one another. This is the case notably in many resource allocation problems, where a single resource can only be used by one agent at a time. How shall a designer of a multi-agent system program its identical agents to behave each in a different way? From a game theoretic perspective, such situations lead to undesirable Nash equilibria. For example consider a resource allocation game in that two players compete for an exclusive access to a single resource. It has three Nash equilibria. The two pure-str…
Large scale power systems are comprised of regional utilities with assets that stream sensor readings in real time. In order to detect cyberattacks, the globally acquired, real time sensor data needs to be analyzed in a centralized fashion. However, owing to operational constraints, such a centralized sharing mechanism turns out to be a major obstacle. In this paper, we propose a blockchain based decentralized framework for detecting coordinated replay attacks with full privacy of sensor data. We develop a Bayesian inference mechanism employing locally reported attack probabilities that is tai…
