MFC-EQ:基于包络 Q 学习的平均场控制用于移动去中心化智能体的编队
MFC-EQ: Mean-Field Control with Envelope Q-Learning for Moving Decentralized Agents in Formation
AI总结:
本文提出 MFC-EQ 框架,利用平均场理论近似智能体动态并采用包络 Q 学习学习偏好无关策略,以解决去中心化 MAiF 中联合状态维度爆炸及双目标偏好适应难题,在性能上超越集中式基准并能处理动态编队变化。
摘要:
我们研究了移动智能体编队(MAiF)的去中心化版本,这是多智能体路径寻找的一种变体,旨在为多个智能体规划无碰撞路径,其双重目标是快速到达目标的同时维持期望的编队。智能体必须在部分观测和有限通信的条件下平衡这些目标。编队维持依赖于所有智能体的联合状态,其维度随智能体数量呈指数级增长,导致学习过程难以处理。此外,学习一个能够适应这两个目标不同线性偏好的单一策略也带来了重大挑战。在本文中,我们提出了基于包络 Q 学习的平均场控制(MFC-EQ),一种针对该双目标多智能体问题的可扩展且适应性强的学习框架。我们利用平均场理论来近似所有智能体的动态,同时通过包络 Q 学习学习一个通用的偏好无关策略。我们在众多实例上对 MFC-EQ 的实证评估表明,它优于最先进的集中式 MAiF 基准。此外,MFC-EQ 能有效处理期望编队动态变化的更复杂场景——这是现有 MAiF 规划器无法应对的挑战。
英文摘要:
We study a decentralized version of Moving Agents in Formation (MAiF), a variant of Multi-Agent Path Finding aiming to plan collision-free paths for multiple agents with the dual objectives of reaching their goals quickly while maintaining a desired formation. The agents must balance these objectives under conditions of partial observation and limited communication. The formation maintenance depends on the joint state of all agents, whose dimensionality increases exponentially with the number of agents, rendering the learning process intractable. Additionally, learning a single policy that can accommodate different linear preferences for these two objectives presents a significant challenge. In this paper, we propose Mean-Field Control with Envelop $Q$-learning (MFC-EQ), a scalable and adaptable learning framework for this bi-objective multi-agent problem. We approximate the dynamics of all agents using mean-field theory while learning a universal preference-agnostic policy through envelop $Q$-learning. Our empirical evaluation of MFC-EQ across numerous instances shows that it outperforms state-of-the-art centralized MAiF baselines. Furthermore, MFC-EQ effectively handles more complex scenarios where the desired formation changes dynamically -- a challenge that existing MAiF planners cannot address.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。