多智能体强化学习的最大互信息框架
A Maximum Mutual Information Framework for Multi-Agent Reinforcement Learning
摘要:
在本文中,我们提出了一个最大互信息(MMI)框架的多智能体强化学习(MARL),使多个代理学习协调的行为,通过正规化的累积回报与动作之间的互信息。通过引入一个潜在的变量,以诱导非零的互信息之间的行动和应用变分的界限,我们得到一个听话的下限所考虑的MMI正则化的目标函数。应用策略迭代来最大化导出的下界,我们提出了一个实用的算法命名为变分最大互信息多智能体行动者-评论家(VM 3-AC),它遵循集中学习与分散执行(CTDE)。我们评估VM 3-AC几个游戏需要协调,数值结果表明,VM 3-AC优于MADDPG和其他MARL算法在多智能体任务需要协调。
英文摘要:
In this paper, we propose a maximum mutual information (MMI) framework for multi-agent reinforcement learning (MARL) to enable multiple agents to learn coordinated behaviors by regularizing the accumulated return with the mutual information between actions. By introducing a latent variable to induce nonzero mutual information between actions and applying a variational bound, we derive a tractable lower bound on the considered MMI-regularized objective function. Applying policy iteration to maximize the derived lower bound, we propose a practical algorithm named variational maximum mutual information multi-agent actor-critic (VM3-AC), which follows centralized learning with decentralized execution (CTDE). We evaluated VM3-AC for several games requiring coordination, and numerical results show that VM3-AC outperforms MADDPG and other MARL algorithms in multi-agent tasks requiring coordination.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。