arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2401.07056cs.AIcs.MA
收藏

Aquarium:一个通过多智能体强化学习算法探索捕食者-被捕食者动力学的综合框架

Aquarium: A Comprehensive Framework for Exploring Predator-Prey Dynamics through Multi-Agent Reinforcement Learning Algorithms

Institute of Informatics, LMU Munich(慕尼黑大学信息学研究所);Thomas Lord Department of Computer Science, University of Southern California(南加州大学托马斯·洛德计算机科学系)

Michael Kölle, Yannick Erpelding, Fabian Ritz, Thomy Phan, Steffen Illium, Claudia Linnhoff-Popien

AI总结:

本文提出开源多智能体强化学习环境 Aquarium,用于研究捕食者-被捕食者动力学,支持与 PettingZoo 集成和高度可定制配置,并通过 PPO 实验证明参数共享优于个体学习。

摘要:

多智能体强化学习的最新进展推动了在模拟环境中对智能体之间复杂交互进行建模的研究。其中,捕食者-被捕食者动力学引起了广泛关注,并且已有多种仿真被定制以适应特定需求。为了避免进一步耗时耗力的开发工作,我们提出了 Aquarium,这是一个用于捕食者-被捕食者交互的综合多智能体强化学习环境,可用于研究涌现行为。Aquarium 是开源的,并提供与 PettingZoo 框架的无缝集成,从而能够借助经过验证的算法实现快速启动。该环境具有基于物理的智能体运动,运行在一个二维、边缘环绕的平面上。智能体-环境交互(观测、动作、奖励)以及环境设置(智能体速度、猎物繁殖、捕食者饥饿等)均可完全自定义。除了资源高效的可视化之外,Aquarium 还支持录制视频文件,从而提供对智能体行为的直观理解。为了展示该环境的能力,我们进行了初步研究,使用 PPO 训练多个猎物智能体以躲避捕食者。与文献一致,我们发现个体学习(Individual Learning)的性能不如参数共享(Parameter Sharing),后者显著提升了协调性和样本效率。

英文摘要:

Recent advances in Multi-Agent Reinforcement Learning have prompted the modeling of intricate interactions between agents in simulated environments. In particular, the predator-prey dynamics have captured substantial interest and various simulations been tailored to unique requirements. To prevent further time-intensive developments, we introduce Aquarium, a comprehensive Multi-Agent Reinforcement Learning environment for predator-prey interaction, enabling the study of emergent behavior. Aquarium is open source and offers a seamless integration of the PettingZoo framework, allowing a quick start with proven algorithm implementations. It features physics-based agent movement on a two-dimensional, edge-wrapping plane. The agent-environment interaction (observations, actions, rewards) and the environment settings (agent speed, prey reproduction, predator starvation, and others) are fully customizable. Besides a resource-efficient visualization, Aquarium supports to record video files, providing a visual comprehension of agent behavior. To demonstrate the environment's capabilities, we conduct preliminary studies which use PPO to train multiple prey agents to evade a predator. In accordance to the literature, we find Individual Learning to result in worse performance than Parameter Sharing, which significantly improves coordination and sample-efficiency.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑