Multi-UAV Coverage Path Planning for the Inspection of Large and Complex Structures
A*STAR Institute of Information Research (IR)（新加坡科技研究局信息研究所）；A*STAR Institute of High Performance Computing (IHPC)（新加坡科技研究局高性能计算研究所）；A*STAR Initiative for Artificial Intelligence and Analytics (AI)（新加坡科技研究局人工智能与分析计划）；Department of Mechanical Engineering, Carnegie Mellon University（卡内基梅隆大学机械工程系）
We present a multi-UAV Coverage Path Planning (CPP) framework for the inspection of large-scale, complex 3D structures. In the proposed sampling-based coverage path planning method, we formulate the multi-UAV inspection applications as a multi-agent coverage path planning problem. By combining two NP-hard problems: Set Covering Problem (SCP) and Vehicle Routing Problem (VRP), a Set-Covering Vehicle Routing Problem (SC-VRP) is formulated and subsequently solved by a modified Biased Random Key Genetic Algorithm (BRKGA) with novel, efficient encoding strategies and local improvement heuristics. We test our proposed method for several complex 3D structures with the 3D model extracted from OpenStreetMap. The proposed method outperforms previous methods, by reducing the length of the planned inspection path by up to 48%
This article addresses the problem of Cooperative Coverage Path Planning (C-CPP) for the inspection of complex infrastructures (offline 3D reconstruction) by utilizing multiple Unmanned Autonomous Vehicles (UAVs). The proposed scheme, based on a priori 3D model of the infrastructure under inspection, is able to generate multiple paths for UAVs in order to achieve a complete cooperative coverage in a short time. Initially the infrastructure under inspection is being sliced by horizontal planes, which has the capability of recognizing the branches of the structure and these branches will be hand…
University of Rijeka（里耶卡大学）；University of Southern Denmark（南丹麦大学）
The application of autonomous UAVs to infrastructure inspection tasks provides benefits in terms of operation time reduction, safety, and cost-effectiveness. This paper presents trajectory planning for three-dimensional autonomous multi-UAV volume coverage and visual inspection of infrastructure based on the Heat Equation Driven Area Coverage (HEDAC) algorithm. The method generates trajectories using a potential field and implements distance fields to prevent collisions and to determine UAVs' camera orientation. It successfully achieves coverage during the visual inspection of complex structur…
A*STAR Artificial Intelligence Initiative (A*AI)（新加坡科技研究局人工智能计划（A*AI））；Institute of High Performance Computing（高性能计算研究所）；Carnegie Mellon University（卡内基梅隆大学）
Planning the path to gather the surface information of the target objects is crucial to improve the efficiency of and reduce the overall cost, for visual inspection applications with Unmanned Aerial Vehicles (UAVs). Coverage Path Planning (CPP) problem is often formulated for these inspection applications because of the coverage requirement. Traditionally, researchers usually plan and optimize the viewpoints to capture the surface information first, and then optimize the path to visit the selected viewpoints. In this paper, we propose a novel planning method to directly sample and plan the ins…
Vietnam National University（越南国立大学）；University of Technology Sydney（悉尼科技大学）
In built infrastructure monitoring, an efficient path planning algorithm is essential for robotic inspection of large surfaces using computer vision. In this work, we first formulate the inspection path planning problem as an extended travelling salesman problem (TSP) in which both the coverage and obstacle avoidance were taken into account. An enhanced discrete particle swarm optimization (DPSO) algorithm is then proposed to solve the TSP, with performance improvement by using deterministic initialization, random mutation, and edge exchange. Finally, we take advantage of parallel computing to…
University of Shanghai for Science and Technology（上海理工大学）；Virginia Tech（弗吉尼亚理工大学）
The optical scanning gauges mounted on the robots are commonly used in quality inspection, such as verifying the dimensional specification of sheet structures. Coverage path planning (CPP) significantly influences the accuracy and efficiency of robotic quality inspection. Traditional CPP strategies focus on minimizing the number of viewpoints or traveling distance of robots under the condition of full coverage inspection. The measurement uncertainty when collecting the scanning data is less considered in the free-form surface inspection. To address this problem, a novel CPP method with the opt…
There is a strong demand for covering a large area autonomously by multiple UAVs (Unmanned Aerial Vehicles) supported by a ground vehicle. Limited by UAVs' battery life and communication distance, complete coverage of large areas typically involves multiple take-offs and landings to recharge batteries, and the transportation of UAVs between operation areas by a ground vehicle. In this paper, we introduce a novel large-area-coverage planning framework which collectively optimizes the paths for aerial and ground vehicles. Our method first partitions a large area into sub-areas, each of which a g…
Brazilian Institute of Robotics（巴西机器人研究所）；SENAI CIMATEC（巴西国家工业培训服务西玛特克分校）；Department of Informatics, University of Oslo（奥斯陆大学信息学系）；SENAI Institute for Innovation in Automation（巴西国家工业培训服务自动化创新研究所）；Robotics Innovation Center, DFKI GmbH（德国人工智能研究中心机器人创新中心）
An important open problem in robotic planning is the autonomous generation of 3D inspection paths -- that is, planning the best path to move a robot along in order to inspect a target structure. We recently suggested a new method for planning paths allowing the inspection of complex 3D structures, given a triangular mesh model of the structure. The method differs from previous approaches in its emphasis on generating and considering also plans that result in imperfect coverage of the inspection target. In many practical tasks, one would accept imperfections in coverage if this results in a sub…
Connectivity-Aware UAV Path Planning with Aerial Coverage Maps
Department of ECE, Hong Kong University of Science and Technology, Hong Kong（香港科技大学电子工程系）；Department of EIE, The Hong Kong Polytechnic University, Hong Kong（香港理工大学电子工程系）
Continuous World Coverage Path Planning for Fixed-Wing UAVs using Deep Reinforcement Learning
TUM School of Engineering and Design, Technical University of Munich（慕尼黑技术大学工程与设计学院）；Dept. of Electrical Engineering and Computer Sciences, University of California, Berkeley（加州大学伯克利分校电气工程与计算机科学系）
Technical University of Munich（慕尼黑工业大学）；EURECOM（欧洲通信学院）
Coverage path planning (CPP) is the task of designing a trajectory that enables a mobile agent to travel over every point of an area of interest. We propose a new method to control an unmanned aerial vehicle (UAV) carrying a camera on a CPP mission with random start positions and multiple options for landing positions in an environment containing no-fly zones. While numerous approaches have been proposed to solve similar CPP problems, we leverage end-to-end reinforcement learning (RL) to learn a control policy that generalizes over varying power constraints for the UAV. Despite recent improvem…
University of Genova（热那亚大学）；JPDroni S.r.l.（JPDroni有限责任公司）
Since photovoltaic (PV) plants require periodic maintenance, using Unmanned Aerial Vehicles (UAV) for inspections can help reduce costs. The thermal and visual inspection of PV installations is currently based on UAV photogrammetry. A UAV equipped with a Global Positioning System (GPS) receiver is assigned a flight zone: the UAV will cover it back and forth to collect images to be later composed in an orthomosaic. The UAV typically flies at a height above the ground that is appropriate to ensure that images overlap even in the presence of GPS positioning errors. However, this approach has two …
We study the problem of infrastructure inspection using an Unmanned Aerial Vehicle (UAV) in box girder bridge environments. We consider a scenario where the UAV needs to fully inspect box girder bridges and localize along the bridge surface when standard methods like GPS and optical flow are denied. Our method for overcoming the difficulties of box girder bridges consist of creating local navigation routines, a supervisor, and a planner. The local navigation routines use two 2D Lidars for girder and column flight. For switching between local navigation routines we implement a supervisor which …
A UAV-Mounted Sensor Network for Close-Range Inspection of Wind Turbine Rotor Blades
Department of Computer Science XVII: Robotics Julius-Maximilians Universit\" a t W\" u rzburg W\" u rzburg, Germany；Research；Elektronik-Entwicklung Oktopus GmbH Kiel, Germany
Carniege Mellon University（卡内基梅隆大学）；Massachusetts Institute of Technology（麻省理工学院）
In Japan, inspection of irrigation water canals has been mostly conducted manually. However, the huge demand for more regular inspections as infrastructure ages, coupled with the limited time window available for inspection, has rendered manual inspection increasingly insufficient. With shortened inspection time and reduced labor cost, automated inspection using a combination of unmanned aerial vehicles (UAVs) and ground vehicles (cars) has emerged as an attractive alternative to manual inspection. In this paper, we propose a path planning framework that generates optimal plans for UAVs and ca…
The communication system is a critical part of the system design for the autonomous UAV. It has to address different considerations, including efficiency, reliability and mobility of the UAV. In addition, a multi-UAV system requires a communication system to assist information sharing, task allocation and collaboration in a team of UAVs. In this paper, we review communication solutions for supporting a team of UAVs while considering an application in the power line inspection industry. We provide a review of candidate wireless communication technologies {for supporting communication in UAV app…
UAVs have been widely used in visual inspections of buildings, bridges and other structures. In either outdoor autonomous or semi-autonomous flights missions strong GPS signal is vital for UAV to locate its own positions. However, strong GPS signal is not always available, and it can degrade or fully loss underneath large structures or close to power lines, which can cause serious control issues or even UAV crashes. Such limitations highly restricted the applications of UAV as a routine inspection tool in various domains. In this paper a vision-model-based real-time self-positioning method is …
The Chinese University of Hong Kong（香港中文大学）；Minjiang University（闽江学院）
Automatic crack detection and segmentation play a significant role in the whole system of unmanned aerial vehicle inspections. In this paper, we have implemented a deep learning framework for crack detection based on classical network architectures including Alexnet, VGG, and Resnet. Moreover, inspired by the feature pyramid network architecture, a hierarchical convolutional neural network (CNN) deep learning framework which is efficient in crack segmentation is also proposed, and its performance of it is compared with other state-of-the-art network architecture. We have summarized the existin…
Unmanned Aerial Vehicle (UAV) has already demonstrated its potential in many civilian applications, and the façade inspection is among the most promising ones. In this paper, we focus on enabling the autonomous perception and control of a small UAV for a façade inspection task. Specifically, we consider the perception as a planar object pose estimation problem by simplifying the building structure as concatenation of planes, and the control as an optimal reference tracking control problem. First, a vision based adaptive observer is proposed which can realize stable plane pose estimation under …
This paper presents a distributed approach to provide persistent coverage of an arbitrarily shaped area using heterogeneous coverage of fixed-wing unmanned aerial vehicles (UAVs), and to recover from simultaneous failures of multiple UAVs. The proposed approach discusses level-homogeneous deployment and maintenance of a homogeneous fleet of fixed-wing UAVs given the boundary information and the minimum loitering radius. The UAVs are deployed at different altitude levels to provide heterogeneous coverage and sensing. We use an efficient square packing method to deploy the UAVs, given the minimu…
International Institute of Information Technology, Hyderabad（海得拉巴国际信息技术学院）；Indian Institute of Technology, Kharagpur（印度理工学院卡拉格普尔分校）；University of Petroleum and Energy Studies（石油与能源研究大学）
Unmanned Aerial Vehicle (UAV) based remote sensing system incorporated with computer vision has demonstrated potential for assisting building construction and in disaster management like damage assessment during earthquakes. The vulnerability of a building to earthquake can be assessed through inspection that takes into account the expected damage progression of the associated component and the component's contribution to structural system performance. Most of these inspections are done manually, leading to high utilization of manpower, time, and cost. This paper proposes a methodology to auto…
University of British Columbia（不列颠哥伦比亚大学）；The Chinese University of Hong Kong（香港中文大学）
Unmanned Aerial Vehicles (UAVs) have been implemented for environmental monitoring by using their capabilities of mobile sensing, autonomous navigation, and remote operation. However, in real-world applications, the limitations of on-board resources (e.g., power supply) of UAVs will constrain the coverage of the monitored area and the number of the acquired samples, which will hinder the performance of field estimation and mapping. Therefore, the issue of constrained resources calls for an efficient sampling planner to schedule UAV-based sensing tasks in environmental monitoring. This paper pr…
University of Technology Sydney（悉尼科技大学）；Vietnam National University, Hanoi（河内越南国家大学）；VNU University of Engineering and Technology（越南国家大学工程技术大学）
Unmanned aerial vehicles (UAVs) are being employed in many areas such as photography, emergency, entertainment, defence, agriculture, forestry, mining and construction. Over the last decade, UAV technology has found applications in numerous construction project phases, ranging from site mapping, progress monitoring, building inspection, damage assessments, and material delivery. While extensive studies have been conducted on the advantages of UAVs for various construction-related processes, studies on UAV collaboration to improve the task capacity and efficiency are still scarce. This paper pr…
Singular Perception s.r.l（Singular Perception有限责任公司）；Teseo s.r.l.（Teseo有限责任公司）；University of Genoa（热那亚大学）
Unmanned Aerial Vehicles (UAVs) are becoming increasingly useful for tasks which require the acquisition of data over large areas. The coverage problem, i.e., the problem of periodically visiting all subregions of an area at a desired frequency, is especially interesting because of its practical applications, both in industry and long-term monitoring of areas hit by a natural disaster. We focus here on the latter scenario, and take into consideration its peculiar characteristic, i.e. the a coverage system should be resilient to a changing environment and not be dependent on pre-existing infras…
Autonomous UAV Pipeline Near-proximity Inspection via Disturbance-Aware Predictive Visual Servoing
School of Automation, Key Laboratory of Measurement and Control of CSE, Ministry of Education, Southeast University（自动化学院、CSE测量与控制重点实验室、教育部、东南大学）；Department of Aeronautical and Automotive Engineering, Loughborough University（航空与汽车工程系、罗汉布大学）；Research Centre for Low Altitude Economy and the Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University（低空经济研究中心和航空与航空工程系、香港理工大学）
Virginia Tech（弗吉尼亚理工大学）；University of South Carolina（南卡罗来纳大学）；University of Maryland（马里兰大学）
We study the problem of covering an environment using an Unmanned Aerial Vehicle (UAV) with limited battery capacity. We consider a scenario where the UAV can land on an Unmanned Ground Vehicle (UGV) and recharge the onboard battery. The UGV can also recharge the UAV while transporting the UAV to the next take-off site. We present an algorithm to solve a new variant of the area coverage problem that takes into account this symbiotic UAV and UGV system. The input consists of a set of boustrophedon cells -- rectangular strips whose width is equal to the field-of-view of the sensor on the UAV. Th…
Cooperative Aerial Robot Inspection Challenge: A Benchmark for Heterogeneous Multi-UAV Planning and Lessons Learned
Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）；School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）；School of Artificial Intelligence, Sun Yat-sen University（中山大学人工智能学院）；Zhejiang University（浙江大学）；Department of Mechanical and Automation Engineering, Chinese University of Hong Kong（香港中文大学机械与自动化工程学系）
IIIT-Delhi（德里印度理工学院）；CNLS, Los Alamos National Laboratory（洛斯阿拉莫斯国家实验室非线性研究中心）；Texas A&M University（得克萨斯农工大学）；Loughbrough University（拉夫堡大学）
Low cost Unmanned Aerial Vehicles (UAVs) need multiple refuels to accomplish large area coverage. The number of refueling stations and their placement plays a vital role in determining coverage efficiency. In this paper, we propose the use of a ground-based refueling vehicle (RV) to increase the operational range of a UAV in both spatial and temporal domains. Determining optimal routes for the UAV and RV, and selecting optimized locations for refueling to aid in minimizing coverage time is a challenging problem due to different vehicle speeds, coupling between refueling location placement, and…
Optimized Area Coverage in Disaster Response Utilizing Autonomous UAV Swarm Formations
Tech Hive Labs；Control Systems and Robotics Lab (CSRL)（控制系统与机器人实验室）；National Technical University of Athens（国家技术大学雅典）；Fédération ENAC ISAE-SUPAERO ONERA（ENAC ISAE-SUPAERO ONERA联合会）；AI Innovation Centre（人工智能创新中心）
