arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2503.07397cs.LG
收藏

Q-MARL:一种使用神经消息传递的量子启发式算法,用于大规模多智能体强化学习

Q-MARL: A quantum-inspired algorithm using neural message passing for large-scale multi-agent reinforcement learning

University of Technology Sydney (UTS)(悉尼科技大学)

Kha Vo, Chin-Teng Lin

AI总结:

Q-MARL通过将每个智能体建模为动态邻域中的子图并利用消息传递神经网络进行去中心化学习,实现了支持数千智能体的大规模多智能体强化学习,在训练速度与损失上均优于现有方法。

摘要:

受量子化学中一种基于图的分子性质预测技术——原子在三维空间分子内的位置——的启发,我们提出了Q-MARL,这是一种完全去中心化的学习架构,支持超大规模多智能体强化学习场景,而无需诸如共同奖励或智能体顺序之类的强假设。其关键在于将每个智能体视为相对于其周围智能体的存在,并假定环境会动态变化。因此,在每个时间步中,一个智能体既是其自身邻域的中心,同时也是许多其他智能体的邻居。每种角色都被表述为一个子图,每个子图都被用作训练样本。消息传递神经网络支持局部邻域内全尺度的顶点和边交互,而一个控制子图深度的参数减轻了训练负担。在测试期间,一个智能体的动作会在所有包含它的子图中进行局部集成,从而产生稳健的决策。在其他方法难以管理50个智能体的情况下,Q-MARL可以轻松调度数千个智能体。详细的理论分析证明了改进性和收敛性,并且使用典型的协作与竞争场景进行的仿真显示出显著更快的训练速度和更低的训练损失。

英文摘要:

Inspired by a graph-based technique for predicting molecular properties in quantum chemistry -- atoms' position within molecules in three-dimensional space -- we present Q-MARL, a completely decentralised learning architecture that supports very large-scale multi-agent reinforcement learning scenarios without the need for strong assumptions like common rewards or agent order. The key is to treat each agent as relative to its surrounding agents in an environment that is presumed to change dynamically. Hence, in each time step, an agent is the centre of its own neighbourhood and also a neighbour to many other agents. Each role is formulated as a sub-graph, and each sub-graph is used as a training sample. A message-passing neural network supports full-scale vertex and edge interaction within a local neighbourhood, while a parameter governing the depth of the sub-graphs eases the training burden. During testing, an agent's actions are locally ensembled across all the sub-graphs that contain it, resulting in robust decisions. Where other approaches struggle to manage 50 agents, Q-MARL can easily marshal thousands. A detailed theoretical analysis proves improvement and convergence, and simulations with the typical collaborative and competitive scenarios show dramatically faster training speeds and reduced training losses.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑