多智能体强化学习方法如VDN、QMIX和QTRAN采用集中训练与分散执行（CTDE）框架，在合作和竞争方面显示出良好的效果。然而，在一些多代理场景中，代理的数量和动作集的大小实际上随时间而变化。我们称之为不成形的场景，上面提到的方法都不能令人满意地执行。在本文中，我们提出了一种新的方法称为Unshaped Networks for Multi-Agent Systems（UNMAS），以适应多智能体系统的数量和大小的变化。我们提出了自加权混合网络来分解联合作用值。它对个体数量变化的适应性归因于个体Q值到个体权重的联合动作值的非线性映射。此外，为了解决行动集的变化，每个代理构建一个单独的行动价值网络，该网络由两个流组成，以评估恒定的环境导向子集和变化的单元导向子集。我们评估UNMAS在各种星际争霸II微观管理方案，并比较结果与几个国家的最先进的MARL算法。地雷行动处的优越性体现在其最高的中奖率上，特别是在最困难的3s5z_vs_3s6z情况下。当其他MARL算法失败时，智能体学习执行有效的合作行为。https://sites.google.com/view/unmas提供了动画演示和源代码。
英文摘要：
Multi-agent reinforcement learning methods such as VDN, QMIX, and QTRAN that adopt centralized training with decentralized execution (CTDE) framework have shown promising results in cooperation and competition. However, in some multi-agent scenarios, the number of agents and the size of action set actually vary over time. We call these unshaped scenarios, and the methods mentioned above fail in performing satisfyingly. In this paper, we propose a new method called Unshaped Networks for Multi-Agent Systems (UNMAS) that adapts to the number and size changes in multi-agent systems. We propose the self-weighting mixing network to factorize the joint action-value. Its adaption to the change in agent number is attributed to the nonlinear mapping from each-agent Q value to the joint action-value with individual weights. Besides, in order to address the change in action set, each agent constructs an individual action-value network that is composed of two streams to evaluate the constant environment-oriented subset and the varying unit-oriented subset. We evaluate UNMAS on various StarCraft II micro-management scenarios and compare the results with several state-of-the-art MARL algorithms. The superiority of UNMAS is demonstrated by its highest winning rates especially on the most difficult scenario 3s5z_vs_3s6z. The agents learn to perform effectively cooperative behaviors while other MARL algorithms fail in. Animated demonstrations and source code are provided in https://sites.google.com/view/unmas.
UAV-UGV Cooperative Trajectory Optimization and Task Allocation for Medical Rescue Tasks in Post-Disaster Environments
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（多模态人工智能系统国家重点实验室，自动化研究所，中国科学院）；Vanke School of Public Health, Institute for Healthy China, Tsinghua University（万科公共卫生学院，中国健康研究院，清华大学）；School of Automation, Beijing Institute of Technology（自动化学院，北京理工大学）；School of Civil Engineering, Tsinghua University（土木工程学院，清华大学）
Towards Comprehensive Testing on the Robustness of Cooperative Multi-agent Reinforcement Learning
Beihang University（北京航空航天大学）；Yangzhou Collaborative Innovation Research Institute CO., LTD（扬州协同创新研究院有限公司）；No. 38 Research Institute of CETC（中国电子科技集团公司第三十八研究所）
An Embodied Multiagent Framework Based on Token Communications for Cooperative ISAC
Tsinghua University（清华大学）；State Key Laboratory of Space Network and Communications, Tsinghua University（空间网络与通信国家重点实验室（清华大学））；Beijing National Research Center for Information Science and Technology, Tsinghua University（北京信息科学与技术国家研究中心（清华大学））
