arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2105.08692cs.AI
收藏

基于Coach-Player多智能体强化学习的动态团队组合

Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Composition

The University of Texas at Austin(德克萨斯大学奥斯汀分校);University of Toronto(多伦多大学);Nvidia(英伟达);California Institute of Technology(加州理工学院)

Bo Liu, Qiang Liu, Peter Stone, Animesh Garg, Yuke Zhu, Animashree Anandkumar

摘要:

在现实世界的多智能体系统中,具有不同能力的智能体可以加入或离开,而不会改变团队的总体目标。协调具有这种动态组合的团队是具有挑战性的:最佳团队策略随组合而变化。我们提出COPA,教练球员的框架来解决这个问题。我们假设教练对环境有全局的看法,并通过分配个人策略来协调只有局部看法的球员。具体而言,我们1)采用注意力机制的教练和球员; 2)提出了一个可变的目标,定期学习;和3)设计了一个自适应的沟通方法,让教练决定何时与球员沟通。我们验证我们的方法上的资源收集任务,救援游戏,和星际争霸微观管理任务。我们展示了zero-shot泛化到新的团队组合。我们的方法实现了可比或更好的性能比设置,所有玩家都有一个完整的环境视图。此外,我们看到,即使教练使用自适应沟通策略沟通的时间只有13%,性能仍然很高。

英文摘要:

In real-world multi-agent systems, agents with different capabilities may join or leave without altering the team's overarching goals. Coordinating teams with such dynamic composition is challenging: the optimal team strategy varies with the composition. We propose COPA, a coach-player framework to tackle this problem. We assume the coach has a global view of the environment and coordinates the players, who only have partial views, by distributing individual strategies. Specifically, we 1) adopt the attention mechanism for both the coach and the players; 2) propose a variational objective to regularize learning; and 3) design an adaptive communication method to let the coach decide when to communicate with the players. We validate our methods on a resource collection task, a rescue game, and the StarCraft micromanagement tasks. We demonstrate zero-shot generalization to new team compositions. Our method achieves comparable or better performance than the setting where all players have a full view of the environment. Moreover, we see that the performance remains high even when the coach communicates as little as 13% of the time using the adaptive communication strategy.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑