HAMMER：通过学习消息的强化学习代理的多级协调
HAMMER: Multi-Level Coordination of Reinforcement Learning Agents via Learned Messaging
摘要：
协作多智能体强化学习（MARL）已经取得了显著的成果，最显著的是利用了深度神经网络的表示学习能力。然而，随着代理人数量的增加，大型集中式方法很快变得不可行，完全分散的方法可能会错过信息共享和协调的重要机会。此外，并非所有代理都是平等的-在某些情况下，单个代理甚至可能没有能力向其他代理发送通信或显式地对其他代理进行建模。本文考虑的情况下，有一个单一的，强大的，\emph{central agent}，可以观察整个观察空间，有多个，低功率的\emph{local agents}，只能接收本地的意见，并不能相互通信。中央代理的工作是根据全局观察来学习需要向不同的本地代理发送什么消息，不是通过集中解决整个问题并发送动作命令，而是通过确定单个代理应该接收什么附加信息，以便它可以做出更好的决策。在这项工作中，我们提出了我们的MARL算法\algo，描述它将是最适用的，并实现它在合作导航和多智能体步行者域。实证结果表明：1）学习的沟通确实提高了系统性能，2）结果推广到异构本地代理，3）结果推广到不同的奖励结构。
英文摘要：
Cooperative multi-agent reinforcement learning (MARL) has achieved significant results, most notably by leveraging the representation-learning abilities of deep neural networks. However, large centralized approaches quickly become infeasible as the number of agents scale, and fully decentralized approaches can miss important opportunities for information sharing and coordination. Furthermore, not all agents are equal -- in some cases, individual agents may not even have the ability to send communication to other agents or explicitly model other agents. This paper considers the case where there is a single, powerful, \emph{central agent} that can observe the entire observation space, and there are multiple, low-powered \emph{local agents} that can only receive local observations and are not able to communicate with each other. The central agent's job is to learn what message needs to be sent to different local agents based on the global observations, not by centrally solving the entire problem and sending action commands, but by determining what additional information an individual agent should receive so that it can make a better decision. In this work we present our MARL algorithm \algo, describe where it would be most applicable, and implement it in the cooperative navigation and multi-agent walker domains. Empirical results show that 1) learned communication does indeed improve system performance, 2) results generalize to heterogeneous local agents, and 3) results generalize to different reward structures.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
