Multi-agent Reinforcement Learning (MARL) is emerging as a key framework for various sequential decision-making and control tasks. Unlike their single-agent counterparts, multi-agent systems necessitate successful cooperation among the agents. The deployment of these systems in real-world scenarios often requires decentralized training, a diverse set of agents, and learning from infrequent environmental reward signals. These challenges become more pronounced under partial observability and the lack of prior knowledge about agent heterogeneity. While notable studies use intrinsic motivation (IM) to address reward sparsity or cooperation in decentralized settings, those dealing with heterogeneity typically assume centralized training, parameter sharing, and agent indexing. To overcome these limitations, we propose the CoHet algorithm, which utilizes a novel Graph Neural Network (GNN) based intrinsic motivation to facilitate the learning of heterogeneous agent policies in decentralized settings, under the challenges of partial observability and reward sparsity. Evaluation of CoHet in the Multi-agent Particle Environment (MPE) and Vectorized Multi-Agent Simulator (VMAS) benchmarks demonstrates superior performance compared to the state-of-the-art in a range of cooperative multi-agent scenarios. Our research is supplemented by an analysis of the impact of the agent dynamics model on the intrinsic motivation module, insights into the performance of different CoHet variants, and its robustness to an increasing number of heterogeneous agents.
From Explicit Communication to Tacit Cooperation:A Novel Paradigm for Cooperative MARL
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Self-Clustering Hierarchical Multi-Agent Reinforcement Learning with Extensible Cooperation Graph
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；CETC Information Science Academy（中国电子科技集团公司信息科学研究院）
Empirical Study on Robustness and Resilience in Cooperative Multi-Agent Reinforcement Learning
State Key Laboratory of Complex & Critical Software Environment（复杂与关键软件环境国家重点实验室）；Beihang University（北京航空航天大学）；Zhongguancun Laboratory（中关村实验室）；Institute of Artificial Intelligence, Peking University（北京大学人工智能研究院）；Institute of data space, Hefei Comprehensive National Science Center（合肥综合国家科学中心数据空间研究院）；Nanyang Technological University（南洋理工大学）
FP3O: Enabling Proximal Policy Optimization in Multi-Agent Cooperation with Parameter-Sharing Versatility
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；The State Key Lab of Brain-Machine Intelligence, Zhejiang University（浙江大学脑机智能国家重点实验室）
Cooperative Policy Learning with Pre-trained Heterogeneous Observation Representations
Microsoft Research（微软研究院）；Beijing University of Posts and Telecommunications（北京邮电大学）；Hong Kong University of Science and Technology（香港科技大学）；Microsoft ARD Incubation Team（微软ARD孵化团队）
Guidelines for Applying RL and MARL in Cybersecurity Applications
Alan Turing Institute（阿兰·图灵研究所）；BAE Systems（BAE系统公司）；Defence Science and Technology Laboratory（国防科学技术实验室）；BMT Group（BMT集团）；Applied Data Science Partners（应用数据科学合作伙伴公司）；Montvieux（蒙维尤公司）；Frazer-Nash Consultancy（弗雷泽-纳什咨询公司）
Inducing Cooperation via Team Regret Minimization based Multi-Agent Deep Reinforcement Learning
Nanyang Technological University（南洋理工大学）；Sun Yat-sen University（中山大学）；King’s College London（伦敦国王学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
MASH: Cooperative-Heterogeneous Multi-Agent Reinforcement Learning for Single Humanoid Robot Locomotion
Faculty of Robot Science and Engineering, Northeastern University（机器人科学与工程学院，东北大学）；Guangdong Key Laboratory of Intelligent Morphing Mechanisms and Adaptive Robotics and School of Intelligence Science and Engineering, the Harbin Institute of Technology Shenzhen（智能变形机制与适应机器人重点实验室和智能科学与工程学院，哈尔滨工业大学深圳）
University of Maryland（马里兰大学）；IISER Bhopal（博帕尔印度科学教育与研究所）；JP Morgan Chase & Co.（摩根大通公司）；DEVCOM Army Research Laboratory（DEVCOM陆军研究实验室）；University of Central Florida（中佛罗里达大学）
Adaptive Value Decomposition: Coordinating a Varying Number of Agents in Urban Systems
State Key Laboratory of General Artificial Intelligence, BIGAI（人工智能通用基础理论国家重点实验室）；Beijing Technology and Business University（北京科技与商业大学）；Peking University（北京大学）
GRSN: Gated Recurrent Spiking Neurons for POMDPs and MARL
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；College of Computer Science, Zhejiang University of Technology（浙江工业大学计算机学院）
Agent-Centric Representations for Multi-Agent Reinforcement Learning
DeepMind；Google Research（谷歌研究院）
以对象为中心的表示最近在处理关系推理任务方面取得了重大进展。通过在神经架构中构建强的以对象为中心的归纳偏差，最近的努力已经提高了这些问题的机器学习算法的泛化和数据效率。涉及关系推理的一个问题类别仍然未得到充分探讨，那就是多智能体强化学习（MARL）。在这里，我们调查是否以对象为中心的表示也是有益的，在充分合作的MARL设置。具体来说，我们研究了两种方法，将代理为中心的归纳偏见到我们的RL算法：1。引入一个以代理为中心的注意力模块，并在代理之间建立显式连接2。添加以代理为中心的无监督预测目标（即不使用动作标签），用作MARL的辅助损失，或作为预训练步骤的基础。我们在Google Research Football环境和DeepMind Lab 2D上评估了这些方法。经验上，以主体为中心的表示学习导致主体之间更复杂的合作策略的出现，以及增强的样本效率和泛化能力。
Mixture of personality improved Spiking actor network for efficient multi-agent cooperation
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Future Technology, University of Chinese Academy of Sciences（中国科学院大学未来技术学院）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
