面向自适应水下多智能体行为的仿生启发式加速强化学习
Towards Bio-inspired Heuristically Accelerated Reinforcement Learning for Adaptive Underwater Multi-Agents Behaviour
AI总结：
针对水下异构自主航行器群的区域覆盖规划问题，本文引入融入PSO仿生启发式的MARL新策略加速训练收敛，将其应用于MSAC算法，减少交互次数即可达到最优性能。
摘要：
本文描述了旨在解决复杂环境下覆盖规划问题的自主多智能体系统（MAS）的协调问题。所考虑的应用场景是在覆盖某一区域的同时检测和识别感兴趣目标。这些任务与空间应用高度相关，同时也在包括水下环境在内的多个领域受到关注，而水下环境正是本研究的重点。在此背景下，覆盖规划传统上被建模为马尔可夫决策过程（MDP），其中需要协调的MAS（即一群异构自主水下航行器）对某一区域进行勘测并搜索目标。该MDP面临若干挑战：环境不确定性、通信约束，以及一系列危险，包括水下环境中时变且不可预测的变化。多智能体强化学习（MARL）算法可利用深度神经网络解决高度非线性问题，并在智能体数量增加时展现出良好的可扩展性。然而，由于MARL算法的学习时间较长，目前水下领域的大多数研究成果仅限于仿真。为此，本研究引入一种新策略，通过融入仿生启发式方法在训练期间引导策略，以加速收敛速度。粒子群优化（PSO）方法因受动物群体行为启发而被选为启发式算法，它能从训练初期就让策略探索动作空间和状态空间的高质量区域，优化探索与利用的权衡。最终得到的智能体只需更少的交互即可达到最优性能。该方法被应用于多智能体软 actor-critic（MSAC）算法，并在连续控制环境下针对二维区域覆盖任务进行了评估。
英文摘要：
This paper describes the problem of coordination of an autonomous Multi-Agent System which aims to solve the coverage planning problem in a complex environment. The considered applications are the detection and identification of objects of interest while covering an area. These tasks, which are highly relevant for space applications, are also of interest among various domains including the underwater context, which is the focus of this study. In this context, coverage planning is traditionally modelled as a Markov Decision Process where a coordinated MAS, a swarm of heterogeneous autonomous underwater vehicles, is required to survey an area and search for objects. This MDP is associated with several challenges: environment uncertainties, communication constraints, and an ensemble of hazards, including time-varying and unpredictable changes in the underwater environment. MARL algorithms can solve highly non-linear problems using deep neural networks and display great scalability against an increased number of agents. Nevertheless, most of the current results in the underwater domain are limited to simulation due to the high learning time of MARL algorithms. For this reason, a novel strategy is introduced to accelerate this convergence rate by incorporating biologically inspired heuristics to guide the policy during training. The PSO method, which is inspired by the behaviour of a group of animals, is selected as a heuristic. It allows the policy to explore the highest quality regions of the action and state spaces, from the beginning of the training, optimizing the exploration/exploitation trade-off. The resulting agent requires fewer interactions to reach optimal performance. The method is applied to the MSAC algorithm and evaluated for a 2D covering area mission in a continuous control environment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
