arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2209.03880cs.MAcs.AIcs.GTcs.LG
收藏

稀疏Graphon平均场博弈的学习

Learning Sparse Graphon Mean Field Games

Technische Universität Darmstadt(达姆施塔特工业大学)

Christian Fabian, Kai Cui, Heinz Koeppl

摘要:

虽然多智能体强化学习(MARL)领域在过去几年中取得了长足的进步,但解决具有大量智能体的系统仍然是一个艰巨的挑战。Graphon平均场游戏(GMFG)使MARL问题的可扩展分析,否则是棘手的。由于图子的数学结构,这种方法仅限于稠密图,这不足以描述许多现实世界的网络,如幂律图。本文介绍了一种新的GMFG公式,称为LPGMFG,它利用了$L^p$图子的图论概念,并提供了一个机器学习工具,以有效和准确地近似稀疏网络问题的解决方案。这尤其包括在各种应用领域中凭经验观察到的幂律网络,并且不能被标准图子捕获。我们推导出理论上的存在性和收敛性保证,并给出实证例子,证明我们的学习方法的准确性与许多代理的系统。此外,我们扩展了在线镜像下降(OMD)学习算法,我们的设置,以加快学习速度,经验表明其能力,并进行理论分析,使用新的概念,平滑的步骤图子。总的来说,我们提供了一个可扩展的,数学上有充分依据的机器学习方法,以解决在众多研究领域中具有重要意义的一大类棘手问题。

英文摘要:

Although the field of multi-agent reinforcement learning (MARL) has made considerable progress in the last years, solving systems with a large number of agents remains a hard challenge. Graphon mean field games (GMFGs) enable the scalable analysis of MARL problems that are otherwise intractable. By the mathematical structure of graphons, this approach is limited to dense graphs which are insufficient to describe many real-world networks such as power law graphs. Our paper introduces a novel formulation of GMFGs, called LPGMFGs, which leverages the graph theoretical concept of $L^p$ graphons and provides a machine learning tool to efficiently and accurately approximate solutions for sparse network problems. This especially includes power law networks which are empirically observed in various application areas and cannot be captured by standard graphons. We derive theoretical existence and convergence guarantees and give empirical examples that demonstrate the accuracy of our learning approach for systems with many agents. Furthermore, we extend the Online Mirror Descent (OMD) learning algorithm to our setup to accelerate learning speed, empirically show its capabilities, and conduct a theoretical analysis using the novel concept of smoothed step graphons. In general, we provide a scalable, mathematically well-founded machine learning approach to a large class of otherwise intractable problems of great relevance in numerous research fields.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑