通过新颖性共享解决去中心化多智能体协同探索问题
Settling Decentralized Multi-Agent Coordinated Exploration by Novelty Sharing
AI总结：
针对去中心化多智能体强化学习中全局新颖性难获取、探索难协同的问题，提出MACE方法，通过共享局部新颖性近似全局新颖性，引入加权互信息作为内在奖励促进协同，在稀疏奖励环境中性能更优。
摘要：
去中心化协作式多智能体强化学习中的探索面临两大挑战。一是全局状态的新颖性无法获取，而局部观测的新颖性存在偏差。二是智能体如何以协同的方式进行探索。为应对这些挑战，我们提出了MACE，一种简单却有效的多智能体协同探索方法。仅通过传递局部新颖性，智能体便可将其他智能体的局部新颖性纳入考量，以近似全局新颖性。此外，我们首次引入加权互信息来衡量一个智能体的动作对其他智能体累积新颖性的影响，并将其转化为事后内在奖励，激励智能体对其他智能体的探索施加更多影响，推动协同探索。实验结果表明，MACE在三个稀疏奖励的多智能体环境中取得了更优的性能。
英文摘要：
Exploration in decentralized cooperative multi-agent reinforcement learning faces two challenges. One is that the novelty of global states is unavailable, while the novelty of local observations is biased. The other is how agents can explore in a coordinated way. To address these challenges, we propose MACE, a simple yet effective multi-agent coordinated exploration method. By communicating only local novelty, agents can take into account other agents' local novelty to approximate the global novelty. Further, we newly introduce weighted mutual information to measure the influence of one agent's action on other agents' accumulated novelty. We convert it as an intrinsic reward in hindsight to encourage agents to exert more influence on other agents' exploration and boost coordinated exploration. Empirically, we show that MACE achieves superior performance in three multi-agent environments with sparse rewards.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
