Towards Autonomous UAV Landing Based on Infrared Beacons and Particle Filtering
Soloviev Rybinsk State Aviation Technical University（索洛维约夫雷宾斯克国立航空技术大学）；NPP SATEK plus（NPP SATEK plus 公司）；Program System Institute of Russian Academy of Sciences（俄罗斯科学院程序系统研究所）；Institute For Systems Analysis of Russian Academy of Sciences（俄罗斯科学院系统分析研究所）
Vsevolod Khithov, Alexander Petrov, Igor Tishchenko, Konstantin Yakovlev
备注：
Final version of the paper as submitted to RiTA 2015 conference
Autonomous fixed-wing UAV landing based on differential GPS is now a mainstream providing reliable and precise landing. But the task still remains challenging when GPS availability is limited like for military UAVs. We discuss a solution of this problem based on computer vision and dot markings along stationary or makeshift runway. We focus our attempts on using infrared beacons along with narrow-band filter as promising way to mark any makeshift runway and utilize particle filtering to fuse both IMU and visual data. We believe that unlike many other vision-based methods this solution is capable of tracking UAV position up to engines stop. System overview, algorithm description and it's evaluation on synthesized sequence along real recorded trajectory is presented.
This paper addresses the problem of developing an algorithm for autonomous ship landing of vertical take-off and landing (VTOL) capable unmanned aerial vehicles (UAVs), using only a monocular camera in the UAV for tracking and localization. Ship landing is a challenging task due to the small landing space, six degrees of freedom ship deck motion, limited visual references for localization, and adversarial environmental conditions such as wind gusts. We first develop a computer vision algorithm which estimates the relative position of the UAV with respect to a horizon reference bar on the landi…
Evidence-Based Landing Site Selection and Vison-Based Landing for UAVs in Unstructured Environments
Aerospace Research Centre, National Research Council Canada（加拿大国家研究委员会航空航天研究中心）；Department of Mechanical and Industrial Engineering, Toronto Metropolitan University（多伦多 Metropolitan 大学机械与工业工程系）
Autonomous and safe landing is important for unmanned aerial vehicles. We present a monocular and stereo image based method for fast and accurate landing zone evaluation for UAVs in various scenarios. Many existing methods rely on Lidar or depth sensor to provide accurate and dense surface reconstruction. We utilize stereo images to evaluate the slope and monocular images to compute homography error. By combining them together, our approach works for both rigid and non-rigid dynamic surfaces. Experiments on many outdoor scenes such as water, grass and roofs, demonstrate the robustness and effe…
The paper discusses an intelligent vision-based control solution for autonomous tracking and landing of Vertical Take-Off and Landing (VTOL) capable Unmanned Aerial Vehicles (UAVs) on ships without utilizing GPS signal. The central idea involves automating the Navy helicopter ship landing procedure where the pilot utilizes the ship as the visual reference for long-range tracking; however, refers to a standardized visual cue installed on most Navy ships called the "horizon bar" for the final approach and landing phases. This idea is implemented using a uniquely designed nonlinear controller int…
A Manipulator-Assisted Multiple UAV Landing System for USV Subject to Disturbance
Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳市人工智能与机器人研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Maastricht University（马斯特里赫特大学）
Unmanned Aerial Vehicles grow more popular by the day and applications for them are crossing boundaries of science and industry, with everything from aerial photography to package delivery to disaster management benefiting from the technology. But before they become commonplace, there are challenges to be solved to make them reliable and safe. The following paper discusses the challenges associated with the precision landing of an Unmanned Aerial Vehicle, including methods for sensing and control and their merits and shortcomings for various applications.
Center for Research in Mathematics CIMAT AC（数学研究中心（CIMAT AC））
Autonomous landing of Unmanned Aerial Vehicles (UAVs) in crowded scenarios is crucial for successful deployment of UAVs in populated areas, particularly in emergency landing situations where the highest priority is to avoid hurting people. In this work, a new visual-based algorithm for identifying Safe Landing Zones (SLZ) in crowded scenarios is proposed, considering a camera mounted on an UAV, where the people in the scene move with unknown dynamics. To do so, a density map is generated for each image frame using a Deep Neural Network, from where a binary occupancy map is obtained aiming to o…
University of Petroleum and Energy Studies（石油与能源研究大学）；International Institute of Information Technology, Hyderabad（海得拉巴国际信息技术学院）
The world we live in is full of technology and with each passing day the advancement and usage of UAVs increases efficiently. As a result of the many application scenarios, there are some missions where the UAVs are vulnerable to external disruptions, such as a ground station's loss of connectivity, security missions, safety concerns, and delivery-related missions. Therefore, depending on the scenario, this could affect the operations and result in the safe landing of UAVs. Hence, this paper presents a heuristic approach towards safe landing of multi-rotor UAVs in the dynamic environments. The…
Harnessing LSTM for Nonlinear Ship Deck Motion Prediction in UAV Autonomous Landing amidst High Sea States
Zhejiang Provincial Engineering Research Centre for Special Aircrafts（浙江省特种飞行器工程研究中心）；Ningbo Institute of Materials Technology & Engineering, CAS（中国科学院宁波材料技术与工程研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Faculty of Electrical Engineering and Computer Science, Ningbo University（宁波大学电气工程与计算机科学学院）
Unmanned Aerial Vehicles (UAVs) equipped with bioradars are a life-saving technology that can enable identification of survivors under collapsed buildings in the aftermath of natural disasters such as earthquakes or gas explosions. However, these UAVs have to be able to autonomously land on debris piles in order to accurately locate the survivors. This problem is extremely challenging as the structure of these debris piles is often unknown and no prior knowledge can be leveraged. In this work, we propose a computationally efficient system that is able to reliably identify safe landing sites an…
The vision of unmanned aerial vehicles is very significant for UAV-related applications such as search and rescue, landing on a moving platform, etc. In this work, we have developed an integrated system for the UAV landing on the moving platform, and the UAV object detection with tracking in the complicated environment. Firstly, we have proposed a robust LoG-based deep neural network for object detection and tracking, which has great advantages in robustness to object scale and illuminations compared with typical deep network-based approaches. Then, we have also improved based on the original …
IAT TU Darmstadt（达姆施塔特工业大学智能自动化研究所(IAT TU Darmstadt)）
In this paper, we introduce a dynamic fiducial marker which can change its appearance according to the spatiotemporal requirements of the visual perception task of a mobile robot using a camera as the sensor. We present a control scheme to dynamically change the appearance of the marker in order to increase the range of detection and to assure a better accuracy on the close range. The marker control takes into account the camera to marker distance (which influences the scale of the marker in image coordinates) to select which fiducial markers to display. Hence, we realize a tight coupling betw…
Tsinghua University（清华大学）；Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）
Selecting safe landing sites in non-cooperative environments is a key step towards the full autonomy of UAVs. However, the existing methods have the common problems of poor generalization ability and robustness. Their performance in unknown environments is significantly degraded and the error cannot be self-detected and corrected. In this paper, we construct a UAV system equipped with low-cost LiDAR and binocular cameras to realize autonomous landing in non-cooperative environments by detecting the flat and safe ground area. Taking advantage of the non-repetitive scanning and high FOV coverage…
Jet Propulsion Laboratory, California Institute of Technology（加州理工学院喷气推进实验室）
The next generation of Mars rotorcrafts requires on-board autonomous hazard avoidance landing. To this end, this work proposes a system that performs continuous multi-resolution height map reconstruction and safe landing spot detection. Structure-from-Motion measurements are aggregated in a pyramid structure using a novel Optimal Mixture of Gaussians formulation that provides a comprehensive uncertainty model. Our multiresolution pyramid is built more efficiently and accurately than past work by decoupling pyramid filling from the measurement updates of different resolutions. To detect the saf…
Unmanned Aerial Vehicles (UAVs) have the potential to be used for many applications in urban environments. However, allowing UAVs to fly above densely populated areas raises concerns regarding safety. One of the main safety issues is the possibility for a failure to cause the loss of navigation capabilities, which can result in the UAV falling/landing in hazardous areas such as busy roads, where it can cause fatal accidents. Current standards, such as the SORA published in 2019, do not consider applicable mitigation techniques to handle this kind of hazardous situations. Consequently, certifyi…
A lot of UAV applications require vertical take-off and landing (VTOL) combined with very long-range or endurance. Transitioning UAVs have been proposed to combine the VTOL capabilities of helicopters with the efficient long-range flight properties of fixed-wing aircraft. But energy is still a bottleneck for many electric long endurance applications. While solar power technology and battery technology have improved a lot, in rougher conditions they still respectively lack the power or total amount of energy required for many real-world situations. In this paper, we introduce the NederDrone, a …
AeroLite-MDNet: Lightweight Multi-task Deviation Detection Network for UAV Landing
Institute of Cyberspace Security, Zhejiang University of Technology（浙江工业大学网络安全学院）；College of Computer Science and Technology, Zhejiang University of Technology（浙江工业大学计算机科学与技术学院）；College of Geoinformatics, Zhejiang University of Technology（浙江工业大学地理信息学院）；Binjiang Institute of Artificial Intelligence, ZJUT（滨江人工智能研究院）；Quzhou Southeast Digital Economic Development Institute（庆州东南数字经济研究院）
Visual-based Safe Landing for UAVs in Populated Areas: Real-time Validation in Virtual Environments
Universidad Autónoma de Ciudad Juárez UACJ（华雷斯自治大学）；Center for Research in Mathematics CIMAT AC（数学研究中心CIMAT AC）；University of Texas at El Paso（德克萨斯大学埃尔帕索分校）
Autonomous Systems Lab, ETH Zurich（苏黎世联邦理工学院自主系统实验室）；CSAIL, MIT（麻省理工学院计算机科学与人工智能实验室）
Full autonomy for fixed-wing unmanned aerial vehicles (UAVs) requires the capability to autonomously detect potential landing sites in unknown and unstructured terrain, allowing for self-governed mission completion or handling of emergency situations. In this work, we propose a perception system addressing this challenge by detecting landing sites based on their texture and geometric shape without using any prior knowledge about the environment. The proposed method considers hazards within the landing region such as terrain roughness and slope, surrounding obstacles that obscure the landing ap…
This paper presents a system for hardware-in-the-loop (HiL) simulation of unmanned aerial vehicle (UAV) control algorithms implemented on a heterogeneous SoC FPGA computing platforms. The AirSim simulator running on a PC and an Arty Z7 development board with a Zynq SoC chip from AMD Xilinx were used. Communication was carried out via a serial USB link. An application for autonomous landing on a specially marked landing strip was selected as a case study. A landing site detection algorithm was implemented on the Zynq SoC platform. This allowed processing a 1280 x 720 @ 60 fps video stream in re…
KIOS Research and Innovation Center of Excellence（KIOS卓越研究与创新中心）；University of Cyprus（塞浦路斯大学）
While Unmanned Aerial Vehicles (UAVs) are increasingly deployed in several missions, their inability of reliable and consistent autonomous landing poses a major setback for deploying such systems truly autonomously. In this paper we present an autonomous UAV landing system for landing on a moving platform. In contrast to existing attempts, the proposed system relies only on the camera sensor, and has been designed as lightweight as possible. The proposed system can be deployed on a low power platform as part of the drone payload, whilst being indifferent to any external communication or any ot…
As UAV technology improves, more uses have been found for these versatile autonomous vehicles, from surveillance to aerial photography, to package delivery, and each of these applications poses unique challenges. This paper implements a solution for one such challenge: To land on a moving target. This problem has been addressed before with varying degrees of success, however, most implementations focus on indoor applications. Outdoor poses greater challenges in the form of variables such as wind and lighting, and outdoor drones are heavier and more susceptible to inertial effects. Our approach…
Université de Toulouse（图卢兹大学）；LAAS-CNRS（国家科学研究中心自动化与系统分析实验室）；ONERA（法国国家航空航天研究院）
To certify UAV operations in populated areas, risk mitigation strategies -- such as Emergency Landing (EL) -- must be in place to account for potential failures. EL aims at reducing ground risk by finding safe landing areas using on-board sensors. The first contribution of this paper is to present a new EL approach, in line with safety requirements introduced in recent research. In particular, the proposed EL pipeline includes mechanisms to monitor learning based components during execution. This way, another contribution is to study the behavior of Machine Learning Runtime Monitoring (MLRM) a…
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）
Heterogeneous teams of mobile robots and UAVs are offering a substantial benefit in an autonomous exploration of the environment. Nevertheless, although joint exploration scenarios for such systems are widely discussed, they are still suffering from low adaptability to changes in external conditions and faults of swarm agents during the UAV docking. We propose a novel vision-based drone swarm docking system for robust landing on a moving platform when one of the agents lost its position signal. The proposed SwarmHawk system relies on vision-based detection for the mobile platform tracking and …
Fly, Track, Land: Infrastructure-less Magnetic Localization for Heterogeneous UAV-UGV Teaming
Department of Engineering, University of Perugia（工程系，佩鲁吉亚大学）；Multi-Robot Systems Group, Faculty of Electrical Engineering, Czech Technical University in Prague（多机器人系统组，电气工程系，布拉格捷克技术大学）；D-ITET Department, ETH Zürich（D-ITET部门，苏黎世联邦理工学院）
With the development of industry, drones are appearing in various field. In recent years, deep reinforcement learning has made impressive gains in games, and we are committed to applying deep reinforcement learning algorithms to the field of robotics, moving reinforcement learning algorithms from game scenarios to real-world application scenarios. We are inspired by the LunarLander of OpenAI Gym, we decided to make a bold attempt in the field of reinforcement learning to control drones. At present, there is still a lack of work applying reinforcement learning algorithms to robot control, the p…
Amazon（亚马逊公司）；West Virginia University（西弗吉尼亚大学）；Microsoft（微软公司）；Gather AI（Gather AI公司）；Mowito（Mowito公司）；Carnegie Mellon University（卡内基梅隆大学）
Many aerial robotic applications require the ability to land on moving platforms, such as delivery trucks and marine research boats. We present a method to autonomously land an Unmanned Aerial Vehicle on a moving vehicle. A visual servoing controller approaches the ground vehicle using velocity commands calculated directly in image space. The control laws generate velocity commands in all three dimensions, eliminating the need for a separate height controller. The method has shown the ability to approach and land on the moving deck in simulation, indoor and outdoor environments, and compared t…
Virginia Tech（弗吉尼亚理工大学）；University of South Carolina（南卡罗来纳大学）；University of Maryland（马里兰大学）
We study the problem of covering an environment using an Unmanned Aerial Vehicle (UAV) with limited battery capacity. We consider a scenario where the UAV can land on an Unmanned Ground Vehicle (UGV) and recharge the onboard battery. The UGV can also recharge the UAV while transporting the UAV to the next take-off site. We present an algorithm to solve a new variant of the area coverage problem that takes into account this symbiotic UAV and UGV system. The input consists of a set of boustrophedon cells -- rectangular strips whose width is equal to the field-of-view of the sensor on the UAV. Th…
The paper discusses a novel vision-based estimation and control approach to enable fully autonomous tracking and landing of vertical take-off and landing (VTOL) capable unmanned aerial vehicles (UAVs) on moving platforms without relying on a GPS signal. A unique feature of the present method is that it accomplishes this task without tracking the landing pad itself; however, by utilizing a standardized visual cue installed normal to the landing pad and parallel to the pilot's/vehicle's line of sight. A computer vision system using a single monocular camera is developed to detect the visual cue …
The University of Manchester（曼彻斯特大学）；Cardiff University（卡迪夫大学）
In recent years, Unmanned Surface Vehicles (USV) have been extensively deployed for maritime applications. However, USV has a limited detection range with sensor installed at the same elevation with the targets. In this research, we propose a cooperative Unmanned Aerial Vehicle - Unmanned Surface Vehicle (UAV-USV) platform to improve the detection range of USV. A floatable and waterproof UAV is designed and 3D printed, which allows it to land on the sea. A catamaran USV and landing platform are also developed. To land UAV on the USV precisely in various lighting conditions, IR beacon detector …
The paper focuses on a heterogeneous swarm of drones to achieve a dynamic landing of formation on a moving robot. This challenging task was not yet achieved by scientists. The key technology is that instead of facilitating each agent of the swarm of drones with computer vision that considerably increases the payload and shortens the flight time, we propose to install only one camera on the leader drone. The follower drones receive the commands from the leader UAV and maintain a collision-free trajectory with the artificial potential field. The experimental results revealed a high accuracy of t…
Risk Assessment for Autonomous Landing in Urban Environments using Semantic Segmentation
Center for Research in Mathematics, CIMAT AC（墨西哥数学研究中心（CIMAT AC））；Centro de Investigación en Matemáticas, A.C.（墨西哥数学研究中心）；Centro de Investigaciones en Óptica, A.C.（墨西哥光学研究中心）；CONAHCYT（墨西哥国家科学技术委员会）
An Experimental Study of Trojan Vulnerabilities in UAV Autonomous Landing
Department of Electrical and Computer Engineering at North Carolina A&T State University（北卡罗来纳A&T州立大学电气与计算机工程系）；University of Alabama（阿拉巴马大学）；Department of Electrical and Computer Engineering, Windsor University（温莎大学电气与计算机工程系）
Unmanned aerial vehicles (UAVs) are often used for navigating dangerous terrains, however they are difficult to pilot. Due to complex input-output mapping schemes, limited perception, the complex system dynamics and the need to maintain a safe operation distance, novice pilots experience difficulties in performing safe landings in obstacle filled environments. In this work we propose a shared autonomy approach that assists novice pilots to perform safe landings on one of several elevated platforms at a proficiency equal to or greater than experienced pilots. Our approach consists of two module…
