基于多智能体强化学习的海洋动力浮标群监测
Multi-Agent Reinforcement Learning for Dynamic Ocean Monitoring by a Swarm of Buoys
摘要:
传统的海洋环境自主监测问题是一个区域覆盖问题,只有多机器人系统才能有效地完成。在本文中,我们专注于机器人群体,通常通过简单的群集行为从一个微妙的,但特设的生物启发的策略组合获得的操作和控制。我们提出了一种新的和结构化的方法,区域覆盖使用多智能体强化学习(MARL),有效地处理非平稳的环境特征。具体来说,我们提出了两种动态区域覆盖方法:(1)基于群的MARL,(2)基于覆盖范围的MARL。前者使用多代理深度确定性策略梯度(MADDPG)方法进行训练,而后者则引入了一个修改后的MADDPG版本,该版本具有内在导致集体行为的奖励函数。这两种方法进行了测试和验证,具有相同的表面积(正方形与矩形)不同的几何形状的区域产生可接受的区域覆盖,并受益于非平稳环境中的结构化学习。这两种方法都比单纯的群集方法有优势。然而,基于覆盖范围的MARL优于基于群的MARL,具有更强的收敛特性,在学习标准和更高的传播代理的区域覆盖。
英文摘要:
Autonomous marine environmental monitoring problem traditionally encompasses an area coverage problem which can only be effectively carried out by a multi-robot system. In this paper, we focus on robotic swarms that are typically operated and controlled by means of simple swarming behaviors obtained from a subtle, yet ad hoc combination of bio-inspired strategies. We propose a novel and structured approach for area coverage using multi-agent reinforcement learning (MARL) which effectively deals with the non-stationarity of environmental features. Specifically, we propose two dynamic area coverage approaches: (1) swarm-based MARL, and (2) coverage-range-based MARL. The former is trained using the multi-agent deep deterministic policy gradient (MADDPG) approach whereas, a modified version of MADDPG is introduced for the latter with a reward function that intrinsically leads to a collective behavior. Both methods are tested and validated with different geometric shaped regions with equal surface area (square vs. rectangle) yielding acceptable area coverage, and benefiting from the structured learning in non-stationary environments. Both approaches are advantageous compared to a naïve swarming method. However, coverage-range-based MARL outperforms the swarm-based MARL with stronger convergence features in learning criteria and higher spreading of agents for area coverage.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。