arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2112.03809cs.MA
收藏

部分可观测的异步多智能体协作挑战

The Partially Observable Asynchronous Multi-Agent Cooperation Challenge

Meng Yao, Qiyue Yin, Jun Yang, Tongtong Yu, Shengqi Shen, Junge Zhang, Bin Liang, Kaiqi Huang

摘要:

多智能体强化学习(MARL)因其在各个领域的应用而受到越来越多的关注。为了满足现实世界的要求,研究者们对它的部分可观测和合作设置给予了很大的关注。为了测试不同算法的性能,设计了标准化的环境,如星际争霸多智能体挑战赛,这是最成功的MARL基准之一。据我们所知,当前的大多数环境都是同步的,其中代理以相同的速度执行操作。然而,异构Agent通常有自己的动作空间,不能保证不同Agent的动作具有相同的执行周期,这就导致了多Agent的异步协作。从战争游戏,从现实世界的环境中抽象的两个军队之间的对抗游戏的启发,我们提出了第一部分可观察的异步多智能体合作的挑战(POAC)的MARL社区。具体来说,POAC支持两个异构代理团队相互战斗,其中代理根据自己的观察选择行动,并与盟友异步合作。此外,POAC是一个轻量级、灵活、易用的环境,可以由用户配置以满足不同的实验需求,如自我游戏模型、人类-AI模型等。与我们的基准测试一起,我们提供了六个不同难度的游戏场景,内置基于规则的AI作为对手。最后,由于大多数MARL算法是为同步代理设计的,我们修改了几个代表,以满足异步设置,相对较差的实验结果验证了POAC的挑战。源代码在\url{http://turingai.ia.ac.cn/data\_center/show}中发布。

英文摘要:

Multi-agent reinforcement learning (MARL) has received increasing attention for its applications in various domains. Researchers have paid much attention on its partially observable and cooperative settings for meeting real-world requirements. For testing performance of different algorithms, standardized environments are designed such as the StarCraft Multi-Agent Challenge, which is one of the most successful MARL benchmarks. To our best knowledge, most of current environments are synchronous, where agents execute actions in the same pace. However, heterogeneous agents usually have their own action spaces and there is no guarantee for actions from different agents to have the same executed cycle, which leads to asynchronous multi-agent cooperation. Inspired from the Wargame, a confrontation game between two armies abstracted from real world environment, we propose the first Partially Observable Asynchronous multi-agent Cooperation challenge (POAC) for the MARL community. Specifically, POAC supports two teams of heterogeneous agents to fight with each other, where an agent selects actions based on its own observations and cooperates asynchronously with its allies. Moreover, POAC is a light weight, flexible and easy to use environment, which can be configured by users to meet different experimental requirements such as self-play model, human-AI model and so on. Along with our benchmark, we offer six game scenarios of varying difficulties with the built-in rule-based AI as opponents. Finally, since most MARL algorithms are designed for synchronous agents, we revise several representatives to meet the asynchronous setting, and the relatively poor experimental results validate the challenge of POAC. Source code is released in \url{http://turingai.ia.ac.cn/data\_center/show}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑