arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2503.09400cs.MA
收藏

网络通信用于均场控制中的去中心化协作智能体

Networked Communication for Decentralised Cooperative Agents in Mean-Field Control

University of Oxford(牛津大学)

Patrick Benjamin, Alessandro Abate

AI总结:

本文提出网络通信用于均场控制中的去中心化协作智能体,证明其能更快提高社会福利。

摘要:

均场框架已被用于解决涉及非常大对称匿名智能体群体的问题,这些问题可能通过其他方法难以处理。尽管均场控制(MFC)问题相比均场博弈(MFG)问题受到较少关注,但前者可能作为工程大规模集体行为工具更具实用性。最近,去中心化通信算法已被引入MFGs,为学习速度和鲁棒性带来了好处。受此启发,我们引入网络通信到MFC中——其中群体可能有更广泛的激励去通信——特别是针对去中心化智能体从单次非回合运行的实证系统中在线学习的设置。我们将最近的MFG算法适应到这一新设置中,并贡献了一个新的子例程,使网络智能体能够从其本地邻域估计全局平均奖励。先前对MFGs中去中心化通信的理论分析无法简单扩展到MFC。因此,我们贡献了新的理论,证明在MFC中网络通信方案使智能体比独立和集中学习两种典型替代架构更快提高社会福利。我们提供了支持这一新结果的不同合作博弈类别的实验,并还给出了关于通信轮次数量和通信故障鲁棒性的多次消融研究和附加实验。

英文摘要:

The mean-field framework has been used to find approximate solutions to problems involving very large populations of symmetric, anonymous agents, which may be intractable by other methods. The cooperative mean-field control (MFC) problem has received less attention than the non-cooperative mean-field game (MFG), despite the former potentially being more useful as a tool for engineering large-scale collective behaviours. Decentralised communication algorithms have recently been introduced to MFGs, giving benefits to learning speed and robustness. Inspired by this, we introduce networked communication to MFC - where populations arguably have broader incentive to communicate - and in particular to the setting where decentralised agents learn online from a single, non-episodic run of the empirical system. We adapt recent MFG algorithms to this new setting, as well as contributing a novel sub-routine allowing networked agents to estimate the global average reward from their local neighbourhood. Previous theoretical analysis of decentralised communication in MFGs does not extend trivially to MFC. We therefore contribute new theory proving that in MFC the networked communication scheme allows agents to increase social welfare faster than under both of the two typical alternative architectures, namely independent and centralised learning. We provide experiments that support this new result across different classes of cooperative game, and also give numerous ablation studies and additional experiments concerning numbers of communication round and robustness to communication failures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑