QGNN：基于图神经网络的价值函数分解
QGNN: Value Function Factorisation with Graph Neural Networks
AI总结：
本文提出QGNN，首个基于图神经网络的价值函数分解方法，通过多层消息传递和置换不变混合器提升多智能体强化学习中的合作行为，在多个基准上优于现有方法。
摘要：
在多智能体强化学习中，使用全局目标函数是激励智能体间合作的有力工具。然而，使用全局奖励训练单个智能体并非样本高效，因为全局奖励不一定与智能体的个体行为相关。这一问题可以通过将全局价值函数分解为局部价值函数来解决。该领域的早期工作仅基于局部信息对局部价值函数进行分解。近期研究表明，同时提供局部信息和全局状态的编码可以促进合作行为。在本文中，我们提出QGNN，这是首个使用基于图神经网络（GNN）模型的价值函数分解方法。QGNN的多层消息传递架构比先前工作中的模型提供了更强的表示复杂度，使其能够产生更有效的分解。QGNN还引入了一种置换不变的混合器，即使在参数显著减少的情况下，也能匹配其他方法的性能。我们针对多个基线方法评估了我们的方法，包括QMIX-Att、GraphMIX、QMIX、VDN以及混合架构。我们的实验包括星际争霸（Starcraft，信用分配的标准基准）、估计游戏（Estimate Game，一个显式建模智能体间依赖关系的自定义环境）以及联盟结构生成（Coalition Structure Generation，一个具有现实应用的基础问题）。结果表明，QGNN在价值函数分解基线上持续优于现有最先进方法。
英文摘要：
In multi-agent reinforcement learning, the use of a global objective is a powerful tool for incentivising cooperation. Unfortunately, it is not sample-efficient to train individual agents with a global reward, because it does not necessarily correlate with an agent's individual actions. This problem can be solved by factorising the global value function into local value functions. Early work in this domain performed factorisation by conditioning local value functions purely on local information. Recently, it has been shown that providing both local information and an encoding of the global state can promote cooperative behaviour. In this paper we propose QGNN, the first value factorisation method to use a graph neural network (GNN) based model. The multi-layer message passing architecture of QGNN provides more representational complexity than models in prior work, allowing it to produce a more effective factorisation. QGNN also introduces a permutation invariant mixer which is able to match the performance of other methods, even with significantly fewer parameters. We evaluate our method against several baselines, including QMIX-Att, GraphMIX, QMIX, VDN, and hybrid architectures. Our experiments include Starcraft, the standard benchmark for credit assignment; Estimate Game, a custom environment that explicitly models inter-agent dependencies; and Coalition Structure Generation, a foundational problem with real-world applications. The results show that QGNN outperforms state-of-the-art value factorisation baselines consistently.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
