Unmanned Aerial Vehicle (UAV) has already demonstrated its potential in many civilian applications, and the façade inspection is among the most promising ones. In this paper, we focus on enabling the autonomous perception and control of a small UAV for a façade inspection task. Specifically, we consider the perception as a planar object pose estimation problem by simplifying the building structure as concatenation of planes, and the control as an optimal reference tracking control problem. First, a vision based adaptive observer is proposed which can realize stable plane pose estimation under very mild observation conditions. Second, a model predictive controller is designed to achieve stable tracking and smooth transition in a multi-plane scenario, while the persistent excitation (PE) condition of the observer and the maneuver constraints of the UAV are satisfied. The proposed autonomous plane pose estimation and plane tracking methods are tested in both simulation and practical building fasçade inspection scenarios, which demonstrate their effectiveness and practicability.
University of Technology Sydney（悉尼科技大学）；University of Engineering and Technology, Vietnam National University（越南国立大学工程技术大学）
This paper presents a real-time control system for surface inspection using multiple unmanned aerial vehicles (UAVs). The UAVs are coordinated in a specific formation to collect data of the inspecting objects. The communication platform for data transmission is based on the Internet of Things (IoT). In the proposed architecture, the UAV formation is established via using the angle-encoded particle swarm optimisation to generate an inspecting path and redistribute it to each UAV where communication links are embedded with an IoT board for network and data processing capabilities. Data collected…
Cooperative Aerial Robot Inspection Challenge: A Benchmark for Heterogeneous Multi-UAV Planning and Lessons Learned
Robotics Institute, Carnegie Mellon University（卡内基梅隆大学机器人研究所）；School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）；School of Artificial Intelligence, Sun Yat-sen University（中山大学人工智能学院）；Zhejiang University（浙江大学）；Department of Mechanical and Automation Engineering, Chinese University of Hong Kong（香港中文大学机械与自动化工程学系）
In this paper, we propose an algorithm for the formation of multiple UAVs used in vision-based inspection of infrastructure. A path planning algorithm is first developed by using a variant of the particle swarm optimisation, named theta-PSO, to generate a feasible path for the overall formation configuration taken into account the constraints for visual inspection. Here, we introduced a cost function that includes various constraints on flight safety and visual inspection. A reconfigurable topology is then added based on the use of intermediate waypoints to allow the formation to avoid collisi…
University of Genova（热那亚大学）；JPDroni S.r.l.（JPDroni有限责任公司）
Since photovoltaic (PV) plants require periodic maintenance, using Unmanned Aerial Vehicles (UAV) for inspections can help reduce costs. The thermal and visual inspection of PV installations is currently based on UAV photogrammetry. A UAV equipped with a Global Positioning System (GPS) receiver is assigned a flight zone: the UAV will cover it back and forth to collect images to be later composed in an orthomosaic. The UAV typically flies at a height above the ground that is appropriate to ensure that images overlap even in the presence of GPS positioning errors. However, this approach has two …
Unmanned Aerial Vehicles (UAVs) have recently shown great performance collecting visual data through autonomous exploration and mapping in building inspection. Yet, the number of studies is limited considering the post processing of the data and its integration with autonomous UAVs. These will enable huge steps onward into full automation of building inspection. In this regard, this work presents a decision making tool for revisiting tasks in visual building inspection by autonomous UAVs. The tool is an implementation of fine-tuning a pretrained Convolutional Neural Network (CNN) for surface c…
We study the problem of infrastructure inspection using an Unmanned Aerial Vehicle (UAV) in box girder bridge environments. We consider a scenario where the UAV needs to fully inspect box girder bridges and localize along the bridge surface when standard methods like GPS and optical flow are denied. Our method for overcoming the difficulties of box girder bridges consist of creating local navigation routines, a supervisor, and a planner. The local navigation routines use two 2D Lidars for girder and column flight. For switching between local navigation routines we implement a supervisor which …
The communication system is a critical part of the system design for the autonomous UAV. It has to address different considerations, including efficiency, reliability and mobility of the UAV. In addition, a multi-UAV system requires a communication system to assist information sharing, task allocation and collaboration in a team of UAVs. In this paper, we review communication solutions for supporting a team of UAVs while considering an application in the power line inspection industry. We provide a review of candidate wireless communication technologies {for supporting communication in UAV app…
UAVs have been widely used in visual inspections of buildings, bridges and other structures. In either outdoor autonomous or semi-autonomous flights missions strong GPS signal is vital for UAV to locate its own positions. However, strong GPS signal is not always available, and it can degrade or fully loss underneath large structures or close to power lines, which can cause serious control issues or even UAV crashes. Such limitations highly restricted the applications of UAV as a routine inspection tool in various domains. In this paper a vision-model-based real-time self-positioning method is …
The Chinese University of Hong Kong（香港中文大学）；Minjiang University（闽江学院）
Automatic crack detection and segmentation play a significant role in the whole system of unmanned aerial vehicle inspections. In this paper, we have implemented a deep learning framework for crack detection based on classical network architectures including Alexnet, VGG, and Resnet. Moreover, inspired by the feature pyramid network architecture, a hierarchical convolutional neural network (CNN) deep learning framework which is efficient in crack segmentation is also proposed, and its performance of it is compared with other state-of-the-art network architecture. We have summarized the existin…
A UAV-Mounted Sensor Network for Close-Range Inspection of Wind Turbine Rotor Blades
Department of Computer Science XVII: Robotics Julius-Maximilians Universit\" a t W\" u rzburg W\" u rzburg, Germany；Research；Elektronik-Entwicklung Oktopus GmbH Kiel, Germany
International Institute of Information Technology, Hyderabad（海得拉巴国际信息技术学院）；Indian Institute of Technology, Kharagpur（印度理工学院卡拉格普尔分校）；University of Petroleum and Energy Studies（石油与能源研究大学）
Unmanned Aerial Vehicle (UAV) based remote sensing system incorporated with computer vision has demonstrated potential for assisting building construction and in disaster management like damage assessment during earthquakes. The vulnerability of a building to earthquake can be assessed through inspection that takes into account the expected damage progression of the associated component and the component's contribution to structural system performance. Most of these inspections are done manually, leading to high utilization of manpower, time, and cost. This paper proposes a methodology to auto…
Carniege Mellon University（卡内基梅隆大学）；Massachusetts Institute of Technology（麻省理工学院）
In Japan, inspection of irrigation water canals has been mostly conducted manually. However, the huge demand for more regular inspections as infrastructure ages, coupled with the limited time window available for inspection, has rendered manual inspection increasingly insufficient. With shortened inspection time and reduced labor cost, automated inspection using a combination of unmanned aerial vehicles (UAVs) and ground vehicles (cars) has emerged as an attractive alternative to manual inspection. In this paper, we propose a path planning framework that generates optimal plans for UAVs and ca…
Unmanned aerial vehicles (UAVs) have become very popular for many military and civilian applications including in agriculture, construction, mining, environmental monitoring, etc. A desirable feature for UAVs is the ability to navigate and perform tasks autonomously with least human interaction. This is a very challenging problem due to several factors such as the high complexity of UAV applications, operation in harsh environments, limited payload and onboard computing power and highly nonlinear dynamics. The work presented in this report contributes towards the state-of-the-art in UAV contro…
National University of Defense Technology（国防科技大学）；KU Leuven（鲁汶大学）；Delft University of Technology（代尔夫特理工大学）
Autonomous flight for UAVs relies on visual information for avoiding obstacles and ensuring a safe collision-free flight. In addition to visual clues, safe UAVs often need connectivity with the ground station. In this paper, we study the synergies between vision and communications for edge computing-enabled UAV flight. By proposing a framework of Edge Computing Assisted Autonomous Flight (ECAAF), we illustrate that vision and communications can interact with and assist each other with the aid of edge computing and offloading, and further speed up the UAV mission completion. ECAAF consists of t…
Visual Servoing NMPC Applied to UAVs for Photovoltaic Array Inspection
University of Alicante（阿利坎特大学）；Instituto de Automática (INAUT)（自动化研究所）；Universidad Nacional de San Juan（圣胡安国立大学）；Centro de Investigaciones de Ciencias Humanas y de la Educación (CICHE)（人文与教育科学研究中心）；Universidad Tecnológica Indoamérica（印多大理工大学）
For the purpose of inspecting power plants, autonomous robots can be built using reinforcement learning techniques. The method replicates the environment and employs a simple reinforcement learning (RL) algorithm. This strategy might be applied in several sectors, including the electricity generation sector. A pre-trained model with perception, planning, and action is suggested by the research. To address optimization problems, such as the Unmanned Aerial Vehicle (UAV) navigation problem, Deep Q-network (DQN), a reinforcement learning-based framework that Deepmind launched in 2015, incorporate…
Korea University（高丽大学）；Mofl Inc.（Mofl公司）；Hallym University（翰林大学）；University of Central Florida（中佛罗里达大学）；Ajou University（亚洲大学）
CCTV-based surveillance using unmanned aerial vehicles (UAVs) is considered a key technology for security in smart city environments. This paper creates a case where the UAVs with CCTV-cameras fly over the city area for flexible and reliable surveillance services. UAVs should be deployed to cover a large area while minimize overlapping and shadow areas for a reliable surveillance system. However, the operation of UAVs is subject to high uncertainty, necessitating autonomous recovery systems. This work develops a multi-agent deep reinforcement learning-based management scheme for reliable indus…
Radio Environmental Maps (REMs) are a powerful tool for enhancing the performance of various communication and networked agents. However, generating REMs is a laborious undertaking, especially in complex 3-Dimensional (3D) environments, such as indoors. To address this issue, we propose a system for autonomous generation of fine-grained REMs of indoor 3D spaces. In the system, multiple small indoor Unmanned Aerial Vehicles (UAVs) are sequentially used for 3D sampling of signal quality indicators. The collected readings are streamlined to a Machine Learning (ML) system for its training and, onc…
Institute for Infocomm Research (IR), A*STAR（新加坡科技研究局资讯通信研究院）；Nanyang Technological University (NTU)（南洋理工大学）；University of Illinois at Urbana-Champaign (UIUC)（伊利诺伊大学厄巴纳-香槟分校）；Singapore University of Technology and Design (SUTD)（新加坡科技设计大学）
Advances in Unmanned Aerial Vehicle (UAV) opens venues for application such as tunnel inspection. Owing to its versatility to fly inside the tunnels, it can quickly identify defects and potential problems related to safety. However, long tunnels, especially with repetitive or uniform structures pose a significant problem for UAV navigation. Furthermore, post-processing visual data from the camera mounted on the UAV is required to generate useful information for the inspection task. In this work, we design a UAV with a single rotating camera to accomplish the task. Compared to other platforms, …
We present a bottleneck analysis tool for designing compute systems for autonomous Unmanned Aerial Vehicles (UAV). The tool provides insights by exploiting the fundamental relationships between various components in the autonomous UAV such as sensor, compute, body dynamics. To guarantee safe operation while maximizing the performance (e.g., velocity) of the UAV, the compute, sensor, and other mechanical properties must be carefully designed (or selected). The goal of our proposed tool is to provide a visual model which aids system architects to understand optimal compute design (or selection) …
The condition assessment of road surfaces is essential to ensure their serviceability while still providing maximum road traffic safety. This paper presents a robust stereo vision system embedded in an unmanned aerial vehicle (UAV). The perspective view of the target image is first transformed into the reference view, and this not only improves the disparity accuracy, but also reduces the algorithm's computational complexity. The cost volumes generated from stereo matching are then filtered using a bilateral filter. The latter has been proved to be a feasible solution for the functional minimi…
Unmanned aerial vehicles (UAV) are commonly used for missions in unknown environments, where an exact mathematical model of the environment may not be available. This paper provides a framework for using reinforcement learning to allow the UAV to navigate successfully in such environments. We conducted our simulation and real implementation to show how the UAVs can successfully learn to navigate through an unknown environment. Technical aspects regarding to applying reinforcement learning algorithm to a UAV system and UAV flight control were also addressed. This will enable continuing research…
University of Technology Sydney（悉尼科技大学）；Vietnam National University, Hanoi（河内越南国家大学）；VNU University of Engineering and Technology（越南国家大学工程技术大学）
Unmanned aerial vehicles (UAVs) are being employed in many areas such as photography, emergency, entertainment, defence, agriculture, forestry, mining and construction. Over the last decade, UAV technology has found applications in numerous construction project phases, ranging from site mapping, progress monitoring, building inspection, damage assessments, and material delivery. While extensive studies have been conducted on the advantages of UAVs for various construction-related processes, studies on UAV collaboration to improve the task capacity and efficiency are still scarce. This paper pr…
Huawei Turkey R&D Center（华为土耳其研发中心）；Istanbul University – Cerrahpasa（伊斯坦布尔大学-塞拉赫帕萨分校）
In this paper, a combat Unmanned Air Vehicle (UAV) is modeled in the simulation environment. The rotary wing UAV is successfully performed various tasks such as locking on the targets, tracking, and sharing the relevant data with surrounding vehicles. Different software technologies such as API communication, ground control station configuration, autonomous movement algorithms, computer vision, and deep learning are employed.
This paper proposes and benchmarks two tether-based motion primitives for tethered UAVs to execute autonomous flight with proprioception only. Tethered UAVs have been studied mainly due to power and safety considerations. Tether is either not included in the UAV motion (treated same as free-flying UAV) or only in terms of station-keeping and high-speed steady flight. However, feedback from and control over the tether configuration could be utilized as a set of navigational tools for autonomous flight, especially in GPS-denied environments and without vision-based exteroception. In this work, t…
Autonomous UAV Pipeline Near-proximity Inspection via Disturbance-Aware Predictive Visual Servoing
School of Automation, Key Laboratory of Measurement and Control of CSE, Ministry of Education, Southeast University（自动化学院、CSE测量与控制重点实验室、教育部、东南大学）；Department of Aeronautical and Automotive Engineering, Loughborough University（航空与汽车工程系、罗汉布大学）；Research Centre for Low Altitude Economy and the Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University（低空经济研究中心和航空与航空工程系、香港理工大学）
KIOS Research and Innovation Center of Excellence（KIOS卓越研究与创新中心）；University of Cyprus（塞浦路斯大学）
While Unmanned Aerial Vehicles (UAVs) are increasingly deployed in several missions, their inability of reliable and consistent autonomous landing poses a major setback for deploying such systems truly autonomously. In this paper we present an autonomous UAV landing system for landing on a moving platform. In contrast to existing attempts, the proposed system relies only on the camera sensor, and has been designed as lightweight as possible. The proposed system can be deployed on a low power platform as part of the drone payload, whilst being indifferent to any external communication or any ot…
Lviv Polytechnic National University（利沃夫国立理工大学）；Terabee（Terabee公司）；Czech Technical University in Prague（捷克布拉格技术大学）
An application of stereo thermal vision to perform preliminary inspection operations of electrical power lines by a particular class of small Unmanned Aerial Vehicles (UAVs), aka Micro Unmanned Aerial Vehicles (MAVs), is presented in this paper. The proposed hardware and software setup allows the detection of overheated power equipment, one of the major causes of power outages. The stereo vision complements the GPS information by finely detecting the potential source of damage while also providing a measure of the harm extension. The reduced sizes and the light weight of the vehicle enable to …
LiDAR-based Quadrotor for Slope Inspection in Dense Vegetation
University of Hong Kong（香港大学）；Geotechnical Engineering Office（土力工程处）；Civil and Engineering Development Department（土木工程拓展署）；The Government of Hong Kong SAR（香港特别行政区政府）
This paper proposes a novel cooperative system for an Unmanned Aerial Vehicle (UAV) and an Unmanned Ground Vehicle (UGV) which utilizes the UAV not only as a flying sensor but also as a tether attachment device. Two robots are connected with a tether, allowing the UAV to anchor the tether to a structure located at the top of a steep terrain, impossible to reach for UGVs. Thus, enhancing the poor traversability of the UGV by not only providing a wider range of scanning and mapping from the air, but also by allowing the UGV to climb steep terrains with the winding of the tether. In addition, we …
