arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.20326cs.MA
收藏

MARLadona——利用多智能体强化学习实现协作团队配合

MARLadona -- Towards Cooperative Team Play Using Multi-Agent Reinforcement Learning

Zichong Li, Filip Bjelonic, Victor Klemm, Marco Hutter

AI总结:

本文提出去中心化 MARL 训练流程 MARLadona 与开源足球环境,通过改进 GEE 学习协作团队配合,并以 66.8% 胜率战胜 HELIOS。

摘要:

机器人足球因其全部复杂性而构成一项尚未解决的研究挑战。现有解决方案严重依赖人工设计的启发式策略,缺乏鲁棒性和适应性。深度强化学习已在运动、操控和竞技游戏(例如 AlphaZero、OpenAI Five)等多种复杂机器人任务中获得广泛关注,因此成为解决机器人足球问题的一种有前景方案。本文介绍 MARLadona,这是一个去中心化的多智能体强化学习(MARL)训练流程,能够产生具备复杂团队配合行为的智能体,弥补启发式方法的不足。此外,我们创建了一个开源多智能体足球环境。利用我们的 MARL 框架以及作为核心架构的改进型全局实体编码器(GEE),该方法在对阵采用最先进启发式策略的 HELIOS 智能体时取得 66.8% 的胜率。我们还深入分析了策略行为,并利用评论家网络解释智能体的意图。

英文摘要:

Robot soccer, in its full complexity, poses an unsolved research challenge. Current solutions heavily rely on engineered heuristic strategies, which lack robustness and adaptability. Deep reinforcement learning has gained significant traction in various complex robotics tasks such as locomotion, manipulation, and competitive games (e.g., AlphaZero, OpenAI Five), making it a promising solution to the robot soccer problem. This paper introduces MARLadona. A decentralized multi-agent reinforcement learning (MARL) training pipeline capable of producing agents with sophisticated team play behavior, bridging the shortcomings of heuristic methods. Furthermore, we created an open-source multi-agent soccer environment. Utilizing our MARL framework and a modified global entity encoder (GEE) as our core architecture, our approach achieves a 66.8% win rate against HELIOS agent, which employs a state-of-the-art heuristic strategy. In addition, we provided an in-depth analysis of the policy behavior and interpreted the agent's intention using the critic network.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑