arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.02896cs.LGcs.AIcs.MA
收藏

基于增强平均场近似的多智能体通信的去卷积

Depthwise Convolution for Multi-Agent Communication with Enhanced Mean-Field Approximation

Nanjing University(南京大学);Tencent(腾讯);University of New South Wales(新南威尔士大学)

Donghan Xie, Zhi Wang, Chunlin Chen, Daoyi Dong

摘要:

由于部分可观察性和代理之间缺乏准确的实时交互,多代理设置仍然是强化学习(RL)领域的一个根本挑战。在本文中,我们提出了一种新的方法,基于本地通信学习,以解决多代理RL(MARL)的挑战,在大量的代理共存。首先,我们设计了一个新的通信协议,利用dependencyconvolution的能力,有效地提取本地关系和学习本地相邻代理之间的通信。为了促进多智能体协调,我们明确地学习联合行动的效果,采取相邻代理的政策作为输入。其次,我们引入平均场近似到我们的方法,以减少代理相互作用的规模。为了更有效地协调相邻代理的行为,我们通过监督策略校正网络(PRN)来校正实时代理交互,并通过可学习的补偿项来校正近似偏差,从而增强了平均场近似。所提出的方法,使有效的协调,以及优于几个基线方法的自适应交通信号控制(ATSC)的任务和星际争霸II多智能体的挑战(SMAC)。

英文摘要:

Multi-agent settings remain a fundamental challenge in the reinforcement learning (RL) domain due to the partial observability and the lack of accurate real-time interactions across agents. In this paper, we propose a new method based on local communication learning to tackle the multi-agent RL (MARL) challenge within a large number of agents coexisting. First, we design a new communication protocol that exploits the ability of depthwise convolution to efficiently extract local relations and learn local communication between neighboring agents. To facilitate multi-agent coordination, we explicitly learn the effect of joint actions by taking the policies of neighboring agents as inputs. Second, we introduce the mean-field approximation into our method to reduce the scale of agent interactions. To more effectively coordinate behaviors of neighboring agents, we enhance the mean-field approximation by a supervised policy rectification network (PRN) for rectifying real-time agent interactions and by a learnable compensation term for correcting the approximation bias. The proposed method enables efficient coordination as well as outperforms several baseline approaches on the adaptive traffic signal control (ATSC) task and the StarCraft II multi-agent challenge (SMAC).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑