This article presents a deep reinforcement learning-based approach to tackle a persistent surveillance mission requiring a single unmanned aerial vehicle initially stationed at a depot with fuel or time-of-flight constraints to repeatedly visit a set of targets with equal priority. Owing to the vehicle's fuel or time-of-flight constraints, the vehicle must be regularly refueled, or its battery must be recharged at the depot. The objective of the problem is to determine an optimal sequence of visits to the targets that minimizes the maximum time elapsed between successive visits to any target while ensuring that the vehicle never runs out of fuel or charge. We present a deep reinforcement learning algorithm to solve this problem and present the results of numerical experiments that corroborate the effectiveness of this approach in comparison with common-sense greedy heuristics.
Alpen-Adria-Universität Klagenfurt（克拉根福阿尔卑斯-亚得里亚大学）；Institute of Networked and Embedded Systems（网络化与嵌入式系统研究所）
We discuss surveillance with multiple unmanned aerial vehicles (UAV) that minimize idleness (the time between consecutive visits of sensing locations) and constrain latency (the time between capturing data at a sensing location and its arrival at the base station). This is important in persistent surveillance scenarios where sensing locations should not only be visited periodically, but the captured data also should reach the base station in due time even if the area is larger than the communication range. Our approach employs the concept of minimum-latency paths (MLP) to guarantee that the da…
We address the problem of achieving persistent surveillance over an environment by using energy-constrained unmanned aerial vehicles (UAVs), which are supported by unmanned ground vehicles (UGVs) serving as mobile charging stations. Specifically, we plan the trajectories of all vehicles and the charging schedule of UAVs to minimize the long-term maximum age, where age is defined as the time between two consecutive visits to regions of interest in a partitioned environment. We introduce a scalable planning strategy based on 1) creating UAV- UGV teams, 2) decomposing the environment into optimal…
This article surveys recent advancements of strategy designs for persistent robotic surveillance tasks with the focus on stochastic approaches. The problem describes how mobile robots stochastically patrol a graph in an efficient way where the efficiency is defined with respect to relevant underlying performance metrics. We first start by reviewing the basics of Markov chains, which is the primary motion model for stochastic robotic surveillance. Then two main criteria regarding the speed and unpredictability of surveillance strategies are discussed. The central objects that appear throughout …
University of California, Santa Barbara（加州大学圣巴巴拉分校）
We design persistent surveillance strategies for the quickest detection of anomalies taking place in an environment of interest. From a set of predefined regions in the environment, a team of autonomous vehicles collects noisy observations, which a control center processes. The overall objective is to minimize detection delay while maintaining the false alarm rate below a desired threshold. We present joint (i) anomaly detection algorithms for the control center and (ii) vehicle routing policies. For the control center, we propose parallel cumulative sum (CUSUM) algorithms (one for each region…
Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室）；Texas A&M University（德州农工大学）；NASA Ames Research Center（美国国家航空航天局艾姆斯研究中心）；Infoscitex corporation（Infoscitex公司）；Air Force Research Laboratory（美国空军研究实验室）
Persistent monitoring missions require an up-to-date knowledge of the changing state of the underlying environment. UAVs can be gainfully employed to continually visit a set of targets representing tasks (and locations) in the environment and collect data therein for long time periods. The enduring nature of these missions requires the UAV to be regularly recharged at a service station. In this paper, we consider the case in which the service station is not co-located with any of the targets. An efficient monitoring requires the revisit time, defined as the maximum of the time elapsed between …
MIT（麻省理工学院）；Laboratory for Information & Decision Systems（信息与决策系统实验室）；Department of Electrical Engineering and Computer Science（电气工程与计算机科学系）；Department of Civil and Environmental Engineering（土木与环境工程系）；Institute of Data Systems and Society（数据系统与社会研究所）
Signalized intersections in arterial roads result in persistent vehicle idling and excess accelerations, contributing to fuel consumption and CO2 emissions. There has thus been a line of work studying eco-driving control strategies to reduce fuel consumption and emission levels at intersections. However, methods to devise effective control strategies across a variety of traffic settings remain elusive. In this paper, we propose a reinforcement learning (RL) approach to learn effective eco-driving control strategies. We analyze the potential impact of a learned strategy on fuel consumption, CO2…
Institute of Networked and Embedded Systems, University of Klagenfurt（克拉根福大学网络化与嵌入式系统研究所）
Mobile robots, especially unmanned aerial vehicles (UAVs), are of increasing interest for surveillance and disaster response scenarios. We consider the problem of multi-robot persistent surveillance with connectivity constraints where robots have to visit sensing locations periodically and maintain a multi-hop connection to a base station. We formally define several problem instances closely related to multi-robot persistent surveillance with connectivity constraints, i.e., connectivity-constrained multi-robot persistent surveillance (CMPS), connectivity-constrained multi-robot reachability (C…
Vanderbilt University（范德堡大学）；Institute of Software Integrated Systems, Vanderbilt University（范德堡大学软件集成系统研究所）
A desirable property in fault-tolerant controllers is adaptability to system changes as they evolve during systems operations. An adaptive controller does not require optimal control policies to be enumerated for possible faults. Instead it can approximate one in real-time. We present two adaptive fault-tolerant control schemes for a discrete time system based on hierarchical reinforcement learning. We compare their performance against a model predictive controller in presence of sensor noise and persistent faults. The controllers are tested on a fuel tank model of a C-130 plane. Our experimen…
RELAX: Reinforcement Learning Enabled 2D-LiDAR Autonomous System for Parsimonious UAVs
School of Data Science, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）数据科学学院）；University of Chicago（芝加哥大学）；University of California, Los Angeles（加州大学洛杉矶分校）
In this paper we propose a control framework that enables robots to execute tasks persistently, i.e., over time horizons much longer than robots' battery life. This is achieved by ensuring that the energy stored in the batteries of the robots is never depleted. This is framed as a set invariance constraint in an optimization problem whose objective is that of minimizing the difference between the robots' control inputs and nominal control inputs corresponding to the task that is to be executed. We refer to this process as the persistification of a robotic task. Forward invariance of subsets of…
Virginia Tech（弗吉尼亚理工大学）；University of Pennsylvania（宾夕法尼亚大学）
We study the problem of planning paths for a team of robots for visually monitoring an environment. Our work is motivated by surveillance and persistent monitoring applications. We are given a set of target points in a polygonal environment that must be monitored using robots with cameras. The goal is to compute paths for all robots such that every target is visible from at least one path. In its general form, this problem is NP-hard as it generalizes the Art Gallery Problem and the Watchman Route Problem. We study two versions: (i) a geometric version in \emph{street polygons} for which we gi…
