Multi-agent reinforcement learning (MARL) has received increasing attention for its applications in various domains. Researchers have paid much attention on its partially observable and cooperative settings for meeting real-world requirements. For testing performance of different algorithms, standardized environments are designed such as the StarCraft Multi-Agent Challenge, which is one of the most successful MARL benchmarks. To our best knowledge, most of current environments are synchronous, where agents execute actions in the same pace. However, heterogeneous agents usually have their own action spaces and there is no guarantee for actions from different agents to have the same executed cycle, which leads to asynchronous multi-agent cooperation. Inspired from the Wargame, a confrontation game between two armies abstracted from real world environment, we propose the first Partially Observable Asynchronous multi-agent Cooperation challenge (POAC) for the MARL community. Specifically, POAC supports two teams of heterogeneous agents to fight with each other, where an agent selects actions based on its own observations and cooperates asynchronously with its allies. Moreover, POAC is a light weight, flexible and easy to use environment, which can be configured by users to meet different experimental requirements such as self-play model, human-AI model and so on. Along with our benchmark, we offer six game scenarios of varying difficulties with the built-in rule-based AI as opponents. Finally, since most MARL algorithms are designed for synchronous agents, we revise several representatives to meet the asynchronous setting, and the relatively poor experimental results validate the challenge of POAC. Source code is released in \url{http://turingai.ia.ac.cn/data\_center/show}.
Learning for Multi-robot Cooperation in Partially Observable Stochastic Environments with Macro-actions
IBM T. J. Watson Research Center(IBM T. J. 沃森研究中心);Princeton University(普林斯顿大学);Massachusetts Institute of Technology(麻省理工学院);Northeastern University(东北大学)
Emergent Cooperation under Uncertain Incentive Alignment
University of Groningen(格罗宁根大学);ILLC & IvI, University of Amsterdam(阿姆斯特丹大学逻辑、语言与计算研究所及信息学研究所);Vrije Universiteit Brussels(布鲁塞尔自由大学);Utrecht University(乌得勒支大学)
Graph Based Deep Reinforcement Learning Aided by Transformers for Multi-Agent Cooperation
School of Computing, Clemson University(克莱姆森大学计算学院);California State University, Los Angeles(加州州立大学洛杉矶分校);Lincoln Laboratory, Massachusetts Institute of Technology(麻省理工学院林肯实验室)
Harnessing Implicit Cooperation: A Multi-Agent Reinforcement Learning Approach Towards Decentralized Local Energy Markets
Department of Mechanical Engineering, Universidad de los Andes(机械工程系,andes大学);Department of Industrial Engineering, Universidad de los Andes(工业工程系,andes大学)
Cooperative Control of Mobile Robots with Stackelberg Learning
University of Colorado Boulder(科罗拉多大学博尔德分校)
多机器人合作要求代理做出与共同目标一致的决策,而不忽视可能因能力和个人目标的不对称而产生的特定于行动的偏好。为了实现这一目标,我们提出了一种名为SLiCC的方法:Stackelberg Learning in Cooperative Control。SLiCC将问题建模为由Stackelberg双矩阵游戏组成的部分可观察随机游戏,并使用深度强化学习来获得与这些游戏相关的支付矩阵。适当的合作行动,然后选择与派生的Stackelberg均衡。使用一个双机器人协作对象运输问题,我们验证了SLiCC对集中式多智能体Q学习的性能,并证明SLiCC实现了更好的组合效用。