arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2004.00470cs.AIcs.MA
收藏

基于图卷积通信的反事实多智能体强化学习

Counterfactual Multi-Agent Reinforcement Learning with Graph Convolution Communication

University of Virginia(弗吉尼亚大学)

Jianyu Su, Stephen Adams, Peter A. Beling

摘要:

我们考虑一个完全合作的多智能体系统,其中代理合作,以最大限度地提高系统的效用在一个部分可观察的环境。我们提出,多智能体系统必须有能力(1)沟通和理解代理之间的相互作用和(2)正确分配奖励的基础上,个人代理的贡献。相比之下,大多数工作在这种情况下只考虑上述能力之一。在这项研究中,我们开发了一个架构,允许代理之间的通信和定制系统的奖励,为每个单独的代理。我们的架构表示智能体通信通过图卷积和应用现有的信用分配结构,反事实多代理策略梯度(COMA),以协助代理学习反向传播通信。图结构的灵活性,使我们的方法适用于各种多代理系统,例如,动态系统,包括不同数量的代理和静态系统与固定数量的代理。我们评估我们的方法在一系列的任务,证明了优势的婚姻通信与信用分配。在实验中,我们提出的方法产生更好的性能比最先进的方法,包括COMA。此外,我们表明,通信策略为我们提供了系统的合作政策的见解和可解释性。

英文摘要:

We consider a fully cooperative multi-agent system where agents cooperate to maximize a system's utility in a partial-observable environment. We propose that multi-agent systems must have the ability to (1) communicate and understand the inter-plays between agents and (2) correctly distribute rewards based on an individual agent's contribution. In contrast, most work in this setting considers only one of the above abilities. In this study, we develop an architecture that allows for communication among agents and tailors the system's reward for each individual agent. Our architecture represents agent communication through graph convolution and applies an existing credit assignment structure, counterfactual multi-agent policy gradient (COMA), to assist agents to learn communication by back-propagation. The flexibility of the graph structure enables our method to be applicable to a variety of multi-agent systems, e.g. dynamic systems that consist of varying numbers of agents and static systems with a fixed number of agents. We evaluate our method on a range of tasks, demonstrating the advantage of marrying communication with credit assignment. In the experiments, our proposed method yields better performance than the state-of-art methods, including COMA. Moreover, we show that the communication strategies offers us insights and interpretability of the system's cooperative policies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑