多智能体确定性Q学习
Multi-Agent Determinantal Q-Learning
摘要：
集中训练与分散执行已经成为多智能体学习的一个重要范式。虽然实用，目前的方法依赖于限制性的假设，分解集中的价值函数跨代理执行。在本文中，我们通过提出多智能体行列式Q学习来消除这种限制。我们的方法是建立在Q-DPP，一个扩展的行列式点过程（DPP）与分区拟阵约束多智能体设置。Q-DPP促进代理获得不同的行为模型，这允许联合Q-函数的自然分解，而不需要对价值函数或特殊网络架构的\emph{a priori}结构约束。我们证明，Q-DPP概括了主要的解决方案，包括VDN，QMIX，和QTRAN上分散的合作任务。为了有效地从Q-DPP中抽取样本，我们采用了一个现有的具有理论近似保证的逐投影采样器。采样器还有利于探索协调代理，以覆盖多代理训练过程中的状态空间中的正交方向。我们评估我们的算法在各种合作的基准，其有效性已被证明与国家的最先进的相比。
英文摘要：
Centralized training with decentralized execution has become an important paradigm in multi-agent learning. Though practical, current methods rely on restrictive assumptions to decompose the centralized value function across agents for execution. In this paper, we eliminate this restriction by proposing multi-agent determinantal Q-learning. Our method is established on Q-DPP, an extension of determinantal point process (DPP) with partition-matroid constraint to multi-agent setting. Q-DPP promotes agents to acquire diverse behavioral models; this allows a natural factorization of the joint Q-functions with no need for \emph{a priori} structural constraints on the value function or special network architectures. We demonstrate that Q-DPP generalizes major solutions including VDN, QMIX, and QTRAN on decentralizable cooperative tasks. To efficiently draw samples from Q-DPP, we adopt an existing sample-by-projection sampler with theoretical approximation guarantee. The sampler also benefits exploration by coordinating agents to cover orthogonal directions in the state space during multi-agent training. We evaluate our algorithm on various cooperative benchmarks; its effectiveness has been demonstrated when compared with the state-of-the-art.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
