arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2011.01129cs.ROcs.AIcs.LG
收藏

多智能体强化学习在可视性持续监测中的应用

Multi-Agent Reinforcement Learning for Visibility-based Persistent Monitoring

University of Maryland, College Park(马里兰大学帕克分校)

Jingxi Chen, Amrish Baskaran, Zhongshun Zhang, Pratap Tokekar

摘要:

基于可见性的持续监控(VPM)问题旨在为机器人找到一组轨迹(或控制器),以持续监控不断变化的环境。每个机器人都有一个传感器,如摄像头,其视野有限,会被环境中的障碍物阻挡。机器人可能需要相互协调,以确保环境中的任何一点都不会长时间处于不受监控的状态。我们对问题进行建模,使得如果一个点未被监控,则每个时间步都会产生一个惩罚。然而,我们不知道惩罚的动态。我们提出了一种用于VPM问题的多智能体强化学习(MARL)算法。具体来说,我们提出了一个多代理图形注意力近端策略优化(MA-G-PPO)算法,作为输入的所有代理的本地观察结合一个低分辨率的全球地图,学习每个代理的政策。图形注意力允许代理与其他人共享他们的信息,从而产生有效的联合策略。我们的主要重点是了解MARL对VPM问题的有效性。我们调查了五个研究问题,这个更广泛的目标。我们发现,MA-G-PPO是能够学习一个更好的政策比非RL基线在大多数情况下,有效性取决于代理共享信息与对方,和政策学习显示紧急行为的代理。

英文摘要:

The Visibility-based Persistent Monitoring (VPM) problem seeks to find a set of trajectories (or controllers) for robots to persistently monitor a changing environment. Each robot has a sensor, such as a camera, with a limited field-of-view that is obstructed by obstacles in the environment. The robots may need to coordinate with each other to ensure no point in the environment is left unmonitored for long periods of time. We model the problem such that there is a penalty that accrues every time step if a point is left unmonitored. However, the dynamics of the penalty are unknown to us. We present a Multi-Agent Reinforcement Learning (MARL) algorithm for the VPM problem. Specifically, we present a Multi-Agent Graph Attention Proximal Policy Optimization (MA-G-PPO) algorithm that takes as input the local observations of all agents combined with a low resolution global map to learn a policy for each agent. The graph attention allows agents to share their information with others leading to an effective joint policy. Our main focus is to understand how effective MARL is for the VPM problem. We investigate five research questions with this broader goal. We find that MA-G-PPO is able to learn a better policy than the non-RL baseline in most cases, the effectiveness depends on agents sharing information with each other, and the policy learnt shows emergent behavior for the agents.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑