arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2006.06455cs.LGcs.MAstat.ML
收藏

多Agent协作中的个体推断通信学习

Learning Individually Inferred Communication for Multi-Agent Cooperation

Peking University(北京大学)

Ziluo Ding, Tiejun Huang, Zongqing Lu

摘要:

沟通是人类合作的基础。这对多代理人合作也至关重要。然而,现有的工作集中在广播通信,这不仅是不切实际的,而且会导致信息冗余,甚至可能损害学习过程。为了解决这些困难,我们提出了个人推断通信(I2C),一个简单而有效的模型,使代理学习代理-智能体通信的先验。先验知识是通过因果推理学习的,并由前馈神经网络实现,该网络将代理的本地观察映射到有关与谁通信的信念。通过多智能体强化学习中的联合动作-价值函数来推断一个智能体对另一个智能体的影响,并将其量化,以标识智能体-智能体通信的必要性。此外,代理策略被正则化,以更好地利用通信消息。经验上,我们表明,I2C不仅可以减少通信开销,但也提高了性能,在各种多代理合作的情况下,与现有的方法相比。该代码可在https://github.com/PKU-AI-Edge/I2C上获得。

英文摘要:

Communication lays the foundation for human cooperation. It is also crucial for multi-agent cooperation. However, existing work focuses on broadcast communication, which is not only impractical but also leads to information redundancy that could even impair the learning process. To tackle these difficulties, we propose Individually Inferred Communication (I2C), a simple yet effective model to enable agents to learn a prior for agent-agent communication. The prior knowledge is learned via causal inference and realized by a feed-forward neural network that maps the agent's local observation to a belief about who to communicate with. The influence of one agent on another is inferred via the joint action-value function in multi-agent reinforcement learning and quantified to label the necessity of agent-agent communication. Furthermore, the agent policy is regularized to better exploit communicated messages. Empirically, we show that I2C can not only reduce communication overhead but also improve the performance in a variety of multi-agent cooperative scenarios, comparing to existing methods. The code is available at https://github.com/PKU-AI-Edge/I2C.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑