arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2301.05873cs.MA
收藏

团队竞争马尔可夫博弈中的对手感知角色学习

Opponent-aware Role-based Learning in Team Competitive Markov Games

Paramita Koley, Aurghya Maiti, Niloy Ganguly, Sourangshu Bhattacharya

摘要:

多智能体马尔可夫博弈中的团队竞争是多智能体强化学习越来越重要的背景,因为它在建模许多现实生活中的情况下具有普遍的适用性。多智能体行动者-批评者方法是最适合在团队竞争环境中学习最优策略的一类技术,因为它们在学习特定于智能体的批评者函数方面具有灵活性,这些批评者函数也可以从其他智能体学习。在许多现实世界的团队竞争场景中,智能体的角色自然会出现,以帮助团队成员之间的协调和协作。然而,现有的方法学习紧急角色在很大程度上依赖于Q-学习设置,不允许学习代理特定的Q-功能。在本文中,我们提出了RAC,一种新的技术,用于学习在一个团队中的代理的紧急角色是多样的和动态的。在所提出的方法中,代理也受益于预测代理在对手团队中的角色。RAC使用角色编码器和对手角色预测器来学习最优策略。使用2个游戏的实验表明,RAC学习的策略比使用最先进的基线学习的策略获得更高的回报。此外,实验表明,在一个团队中的代理学习不同的和服从意识的政策。

英文摘要:

Team competition in multi-agent Markov games is an increasingly important setting for multi-agent reinforcement learning, due to its general applicability in modeling many real-life situations. Multi-agent actor-critic methods are the most suitable class of techniques for learning optimal policies in the team competition setting, due to their flexibility in learning agent-specific critic functions, which can also learn from other agents. In many real-world team competitive scenarios, the roles of the agents naturally emerge, in order to aid in coordination and collaboration within members of the teams. However, existing methods for learning emergent roles rely heavily on the Q-learning setup which does not allow learning of agent-specific Q-functions. In this paper, we propose RAC, a novel technique for learning the emergent roles of agents within a team that are diverse and dynamic. In the proposed method, agents also benefit from predicting the roles of the agents in the opponent team. RAC uses the actor-critic framework with role encoder and opponent role predictors for learning an optimal policy. Experimentation using 2 games demonstrates that the policies learned by RAC achieve higher rewards than those learned using state-of-the-art baselines. Moreover, experiments suggest that the agents in a team learn diverse and opponent-aware policies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑