arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.05787cs.RO
收藏

基于强化学习的图上团队协作规模扩展研究

Scaling Team Coordination on Graphs with Reinforcement Learning

George Mason University(乔治梅森大学)

Manshi Limbu, Zechen Hu, Xuan Wang, Daigo Shishika, Xuesu Xiao

AI总结:

针对经典多智能体路径规划方法在带风险边的图环境中扩展性差的问题,将其转化为新型MDP,提出两种强化学习范式实现高效团队协作,性能优于经典方法。

摘要:

本文研究强化学习(Reinforcement Learning, RL)技术,用于在图环境中实现团队协作行为,队友间可通过支援行动以集中式方式降低穿越某些风险边的成本。经典方法可通过将原始环境图(Environment Graph, EG)转换为联合状态图(Joint State Graph, JSG)来隐式整合支援行动,从而解决这类非标准多智能体路径规划问题,但这些方法无法很好地扩展到大规模图和大型团队。为解决这一维数灾难问题,我们提出利用强化学习,让智能体以数据驱动的方式学习图遍历和队友支援行为。具体而言,我们通过将带风险边的图上团队协作问题重新表述为具有新型状态空间和动作空间的马尔可夫决策过程(Markov Decision Processes, MDPs),从两种范式研究强化学习的求解方式:第一,我们用强化学习让一组智能体学习如何在单个环境图上以最低成本协作到达目标。实验表明,强化学习可高效求解节点/智能体规模达20/4或25/3的问题,所需时间仅为JSG求解这类复杂问题的一小部分;第二,我们学习了一种适用于任意N节点环境图的通用强化学习策略,可产生高效的支援行为。我们开展了大量实验,并将所提强化学习方法与经典对应方法进行了对比。

英文摘要:

This paper studies Reinforcement Learning (RL) techniques to enable team coordination behaviors in graph environments with support actions among teammates to reduce the costs of traversing certain risky edges in a centralized manner. While classical approaches can solve this non-standard multi-agent path planning problem by converting the original Environment Graph (EG) into a Joint State Graph (JSG) to implicitly incorporate the support actions, those methods do not scale well to large graphs and teams. To address this curse of dimensionality, we propose to use RL to enable agents to learn such graph traversal and teammate supporting behaviors in a data-driven manner. Specifically, through a new formulation of the team coordination on graphs with risky edges problem into Markov Decision Processes (MDPs) with a novel state and action space, we investigate how RL can solve it in two paradigms: First, we use RL for a team of agents to learn how to coordinate and reach the goal with minimal cost on a single EG. We show that RL efficiently solves problems with up to 20/4 or 25/3 nodes/agents, using a fraction of the time needed for JSG to solve such complex problems; Second, we learn a general RL policy for any $N$-node EGs to produce efficient supporting behaviors. We present extensive experiments and compare our RL approaches against their classical counterparts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑