量化环境和种群多样性对多智能体强化学习的影响
Quantifying the effects of environment and population diversity in multi-agent reinforcement learning
摘要:
泛化是多智能体强化学习的一个主要挑战。一个智能体在新的环境中以及与新的合作者互动时表现如何?在本文中,我们调查和量化的泛化和多样性之间的关系在多智能体域。在这里考虑的多代理环境的范围内,程序生成训练水平显着提高代理性能的保持水平。然而,在训练中使用的特定级别上的代理性能有时会因此而下降。为了更好地理解合作者变化的影响,我们的实验引入了一个新的环境不可知的行为多样性的措施。结果表明,种群规模和内在动机都是产生更大的种群多样性的有效方法。反过来,在某些(但不是所有)情况下,与不同的合作伙伴一起训练可以加强代理人的表现。
英文摘要:
Generalization is a major challenge for multi-agent reinforcement learning. How well does an agent perform when placed in novel environments and in interactions with new co-players? In this paper, we investigate and quantify the relationship between generalization and diversity in the multi-agent domain. Across the range of multi-agent environments considered here, procedurally generating training levels significantly improves agent performance on held-out levels. However, agent performance on the specific levels used in training sometimes declines as a result. To better understand the effects of co-player variation, our experiments introduce a new environment-agnostic measure of behavioral diversity. Results demonstrate that population size and intrinsic motivation are both effective methods of generating greater population diversity. In turn, training with a diverse set of co-players strengthens agent performance in some (but not all) cases.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。