面向大规模异构多智能体系统的优先级联赛强化学习
Prioritized League Reinforcement Learning for Large-Scale Heterogeneous Multiagent Systems
AI总结:
针对大规模异构多智能体合作中的非平稳性和智能体数量不平衡问题,提出PHLRL方法,通过建立异构策略联赛和优先级策略梯度优化策略,并在LSMO基准中超越QTRAN和QPLEX等先进方法。
摘要:
大规模异构多智能体系统具有现实世界中的多种真实因素,例如具备不同能力的智能体以及整体系统成本。与同构系统相比,异构系统具有显著的实用优势。然而,它们也给多智能体强化学习带来了挑战,包括解决非平稳问题以及管理不同类型智能体数量不平衡的问题。我们提出了一种优先级异构联赛强化学习(PHLRL)方法来解决大规模异构合作问题。PHLRL记录了智能体在训练期间探索的各种策略,并建立了一个由多样化策略组成的异构联赛,以辅助未来的策略优化。此外,我们设计了一种优先级策略梯度方法,以补偿不同类型智能体数量差异造成的差距。接下来,我们使用Unreal Engine设计了一个名为大规模多智能体作战(LSMO)的大规模异构合作基准,这是一个复杂的双团队竞争场景,需要地面和空中智能体的协作。我们通过实验证明,PHLRL在LSMO中优于包括QTRAN和QPLEX在内的最先进方法。
英文摘要:
Large-scale heterogeneous multiagent systems feature various realistic factors in the real world, such as agents with diverse abilities and overall system cost. In comparison to homogeneous systems, heterogeneous systems offer significant practical advantages. Nonetheless, they also present challenges for multiagent reinforcement learning, including addressing the non-stationary problem and managing an imbalanced number of agents with different types. We propose a Prioritized Heterogeneous League Reinforcement Learning (PHLRL) method to address large-scale heterogeneous cooperation problems. PHLRL maintains a record of various policies that agents have explored during their training and establishes a heterogeneous league consisting of diverse policies to aid in future policy optimization. Furthermore, we design a prioritized policy gradient approach to compensate for the gap caused by differences in the number of different types of agents. Next, we use Unreal Engine to design a large-scale heterogeneous cooperation benchmark named Large-Scale Multiagent Operation (LSMO), which is a complex two-team competition scenario that requires collaboration from both ground and airborne agents. We use experiments to show that PHLRL outperforms state-of-the-art methods, including QTRAN and QPLEX in LSMO.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。