It is an amazing fact that remarkably complex behaviors could emerge from a large collection of very rudimentary dynamical agents through very simple local interactions. However, it still remains elusive on how to design these local interactions among agents so as to achieve certain desired collective behaviors. This paper aims to tackle this challenge and proposes a divide-and-conquer approach to guarantee specified global behaviors through local coordination and control design for multi-agent systems. The basic idea is to decompose the requested global specification into subtasks for each individual agent. It should be noted that the decomposition is not arbitrary. The global specification should be decomposed in such a way that the fulfilment of these subtasks by each individual agent will imply the satisfaction of the global specification as a team. First, it is shown by a counterexample that not all specifications can be decomposed in this sense. Then, a natural follow-up question is what the necessary and sufficient condition should be for the proposed decomposability of a global specification. The main part of the paper is set to answer this question. The case of two cooperative agents is investigated first, and a necessary and sufficient condition is presented and proven. Later on, the result is generalized to the case of arbitrary finite number of agents, and a hierarchical algorithm is proposed, which is shown to be a sufficient condition. Finally, a cooperative control scenario for a team of three robots is developed to illustrate the task decomposition procedure.
Dual Self-Awareness Value Decomposition Framework without Individual Global Max for Cooperative Multi-Agent Reinforcement Learning
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
针对合作多智能体强化学习中IGM原则限制问题，提出双自感知价值分解框架，结合自我策略与 alter ego 价值函数并引入反自我探索机制，实现完全无需IGM的价值分解并在多任务中表现优异。
Cooperative Decentralized Multi-agent Control under Local LTL Tasks and Connectivity Constraints
ACCESS Linnaeus Center, School of Electrical Engineering, KTH Royal Institute of Technology（ACCESS林奈中心，电气工程学院，皇家理工学院）；KTH Centre for Autonomous Systems（KTH自主系统中心）
SIDE: State Inference for Partially Observable Cooperative Multi-Agent Reinforcement Learning
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
HAVEN: Hierarchical Cooperative Multi-Agent Reinforcement Learning with Dual Coordination Mechanism
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Mingling Foresight with Imagination: Model-Based Cooperative Multi-Agent Reinforcement Learning
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Cooperative Multi-Agent Reinforcement Learning with Hypergraph Convolution
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Singapore Management University（新加坡管理大学）；University of Chinese Academy of Sciences（中国科学院大学）
Optimistic ε-Greedy Exploration for Cooperative Multi-Agent Reinforcement Learning
University of Electronic Science and Technology of China（电子科技大学）；Southwest Minzu University（西南民族大学）；Chengdu University of Technology（成都理工大学）；Nanyang Technological University（南洋理工大学）
Conditionally Optimistic Exploration for Cooperative Deep Multi-Agent Reinforcement Learning
Mila - Quebec AI Institute（米拉-魁北克人工智能研究所）；École Polytechnique de Montréal（蒙特利尔综合理工学院）；University of Oxford（牛津大学）；University of Alberta（阿尔伯塔大学）；Université de Montréal（蒙特利尔大学）
Offline Multi-agent Reinforcement Learning via Sequential Score Decomposition
School of Data Science, The Chinese University of Hong Kong, Shenzhen, China（香港中文大学（深圳）数据科学学院）；School of Computer Science, Tongji University, Shanghai, China（同济大学计算机科学学院）；Vector Institute（向量研究所）
Towards Global Optimality in Cooperative MARL with the Transformation And Distillation Framework
Washington University in St. Louis（华盛顿大学圣路易斯分校）；Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）
在协作多智能体强化学习（MARL）中，智能体只能访问部分观测，有效地利用本地信息是至关重要的。在长时间的观察中，智能体可以为队友建立\textit{awareness}，以缓解部分可观察性的问题。然而，以往的MARL方法通常忽略了这种利用的本地信息。为了解决这个问题，我们提出了一个新的框架，多代理\textit{Local INformation Decomposition for Awareness of teammates}（LINDA），代理学习分解本地信息和建立意识的每个队友。我们将意识建模为随机随机变量，并通过最大化意识与相应代理的实际轨迹之间的互信息来执行表示学习以确保意识表示的信息性。LINDA对特定算法是不可知的，可以灵活地集成到不同的MARL方法中。大量实验表明，该框架能够从局部局部观测中学习信息感知，提高协作效率，显著提高学习性能，尤其是在具有挑战性的任务中。
