arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1902.07497cs.MA
收藏

分析合作多智能体强化学习中动作价值网络的分解

Analysing Factorizations of Action-Value Networks for Cooperative Multi-Agent Reinforcement Learning

Jacopo Castellini, Frans A. Oliehoek, Rahul Savani, Shimon Whiteson

AI总结:

本研究实证分析了合作多智能体强化学习中各种网络架构在一次性博弈上的学习能力,量化了不同方法表示价值函数的效果,并识别了阻碍性能的关键原因。

摘要:

近年来,深度强化学习技术被应用于合作多智能体系统,取得了巨大的实证成功。然而,由于缺乏理论洞察,目前尚不清楚所采用的神经网络在学习什么,或者我们应如何增强其学习能力以解决它们失败的问题。在本研究中,我们实证地研究了各种网络架构在一系列一次性博弈上的学习能力。尽管这些博弈很简单,但它们捕捉了多智能体环境中出现的许多关键问题,例如联合动作数量呈指数级增长或缺乏明确的协调机制。我们的结果扩展了文献[4]中的研究,量化了各种方法在表示所需价值函数方面的表现,并帮助我们识别阻碍良好性能的原因,如价值稀疏性或过于严格的协调要求。

英文摘要:

Recent years have seen the application of deep reinforcement learning techniques to cooperative multi-agent systems, with great empirical success. However, given the lack of theoretical insight, it remains unclear what the employed neural networks are learning, or how we should enhance their learning power to address the problems on which they fail. In this work, we empirically investigate the learning power of various network architectures on a series of one-shot games. Despite their simplicity, these games capture many of the crucial problems that arise in the multi-agent setting, such as an exponential number of joint actions or the lack of an explicit coordination mechanism. Our results extend those in [4] and quantify how well various approaches can represent the requisite value functions, and help us identify the reasons that can impede good performance, like sparsity of the values or too tight coordination requirements.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑