arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.12686cs.MAcs.AIcs.GTcs.LG
收藏

在稀疏图上学习平均场博弈:一种混合 Graphex 方法

Learning Mean Field Games on Sparse Graphs: A Hybrid Graphex Approach

Technische Universität Darmstadt(达姆施塔特工业大学)

Christian Fabian, Kai Cui, Heinz Koeppl

AI总结:

本文提出 Graphex 平均场博弈(GXMFG)框架,以克服 Graphon MFG 仅适用于稠密图的局限,并设计了一种利用核心-外围结构的混合学习算法,在合成图和真实网络上验证了其有效性。

摘要:

学习大规模智能体群体的行为是众多研究领域的一项重要任务。尽管多智能体强化学习(MARL)领域在求解这些系统方面取得了显著进展,但针对大量智能体的解决方案往往在计算上仍然不可行,并且缺乏理论保证。平均场博弈(MFGs)解决了这两个问题,并且可以扩展为 Graphon MFGs(GMFGs),以纳入智能体之间的网络结构。尽管 GMFGs 具有诸多优点,但其在现实世界中的适用性受到以下事实的限制:graphon 只能刻画稠密图。由于大多数经验观察到的网络都表现出一定程度的稀疏性,例如幂律图,因此 GMFG 框架不足以刻画这些网络拓扑结构。为此,我们引入了 Graphex MFGs(GXMFGs)这一新概念,它建立在 graphex 这一图论概念之上。Graphex 是稀疏图序列的极限对象,同时还具有其他理想特性,例如小世界性质。由于底层图具有丰富且稀疏的结构,在这些博弈中学习均衡具有挑战性。为应对这些挑战,我们设计了一种针对 GXMFG 设置量身定制的新学习算法。这种混合 graphex 学习方法利用了系统主要由一个高度连通的核心和一个稀疏的外围组成这一特点。在定义系统并提供理论分析之后,我们阐述了我们的学习方法,并在合成图和真实世界网络上展示了其学习能力。这一比较表明,我们的 GXMFG 学习算法成功地将 MFGs 扩展到了一类高度相关的、困难的、现实的学习问题上,而当前的 MARL 和 MFG 方法无法准确解决这些问题。

英文摘要:

Learning the behavior of large agent populations is an important task for numerous research areas. Although the field of multi-agent reinforcement learning (MARL) has made significant progress towards solving these systems, solutions for many agents often remain computationally infeasible and lack theoretical guarantees. Mean Field Games (MFGs) address both of these issues and can be extended to Graphon MFGs (GMFGs) to include network structures between agents. Despite their merits, the real world applicability of GMFGs is limited by the fact that graphons only capture dense graphs. Since most empirically observed networks show some degree of sparsity, such as power law graphs, the GMFG framework is insufficient for capturing these network topologies. Thus, we introduce the novel concept of Graphex MFGs (GXMFGs) which builds on the graph theoretical concept of graphexes. Graphexes are the limiting objects to sparse graph sequences that also have other desirable features such as the small world property. Learning equilibria in these games is challenging due to the rich and sparse structure of the underlying graphs. To tackle these challenges, we design a new learning algorithm tailored to the GXMFG setup. This hybrid graphex learning approach leverages that the system mainly consists of a highly connected core and a sparse periphery. After defining the system and providing a theoretical analysis, we state our learning approach and demonstrate its learning capabilities on both synthetic graphs and real-world networks. This comparison shows that our GXMFG learning algorithm successfully extends MFGs to a highly relevant class of hard, realistic learning problems that are not accurately addressed by current MARL and MFG methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑