arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2209.04808cs.MAmath.OC
收藏

图论均值场控制用于协作多智能体强化学习

Graphon Mean-Field Control for Cooperative Multi-Agent Reinforcement Learning

Yuanquan Hu, Xiaoli Wei, Junji Yan, Hengxi Zhang

AI总结:

本文提出图论均值场控制框架,用于近似具有非均匀相互作用的协作多智能体强化学习,证明其近似顺序为$\mathcal{O}(\frac{1}{\sqrt{N}})$,并展示其在可扩展性上的优势。

摘要:

均值场理论与强化学习的结合已显示出解决大规模控制问题的强大能力,特别是在同质智能体的情况下。为了突破均值场理论的同质性限制,最近的研究兴趣是将图论理论引入均值场范式。在本文中,我们提出了一种图论均值场控制(GMFC)框架,用于近似具有非均匀相互作用的协作多智能体强化学习(MARL),并证明近似顺序为$\mathcal{O}(\frac{1}{\sqrt{N}})$,其中$N$是智能体的数量。通过离散化GMFC的图论指数,我们进一步引入了一个更小的GMFC类别,称为块GMFC,它被证明能够很好地近似协作MARL。我们在几个示例中的实证研究显示,我们的GMFC方法在与最先进的MARL算法相媲美的同时,还具有更好的可扩展性。

英文摘要:

The marriage between mean-field theory and reinforcement learning has shown a great capacity to solve large-scale control problems with homogeneous agents. To break the homogeneity restriction of mean-field theory, a recent interest is to introduce graphon theory to the mean-field paradigm. In this paper, we propose a graphon mean-field control (GMFC) framework to approximate cooperative multi-agent reinforcement learning (MARL) with nonuniform interactions and show that the approximate order is of $\mathcal{O}(\frac{1}{\sqrt{N}})$, with $N$ the number of agents. By discretizing the graphon index of GMFC, we further introduce a smaller class of GMFC called block GMFC, which is shown to well approximate cooperative MARL. Our empirical studies on several examples demonstrate that our GMFC approach is comparable with the state-of-art MARL algorithms while enjoying better scalability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑