Selecting safe landing sites in non-cooperative environments is a key step towards the full autonomy of UAVs. However, the existing methods have the common problems of poor generalization ability and robustness. Their performance in unknown environments is significantly degraded and the error cannot be self-detected and corrected. In this paper, we construct a UAV system equipped with low-cost LiDAR and binocular cameras to realize autonomous landing in non-cooperative environments by detecting the flat and safe ground area. Taking advantage of the non-repetitive scanning and high FOV coverage characteristics of LiDAR, we come up with a dynamic time depth completion algorithm. In conjunction with the proposed self-evaluation method of the depth map, our model can dynamically select the LiDAR accumulation time at the inference phase to ensure an accurate prediction result. Based on the depth map, the high-level terrain information such as slope, roughness, and the size of the safe area are derived. We have conducted extensive autonomous landing experiments in a variety of familiar or completely unknown environments, verifying that our model can adaptively balance the accuracy and speed, and the UAV can robustly select a safe landing site.
Jet Propulsion Laboratory, California Institute of Technology（加州理工学院喷气推进实验室）
The next generation of Mars rotorcrafts requires on-board autonomous hazard avoidance landing. To this end, this work proposes a system that performs continuous multi-resolution height map reconstruction and safe landing spot detection. Structure-from-Motion measurements are aggregated in a pyramid structure using a novel Optimal Mixture of Gaussians formulation that provides a comprehensive uncertainty model. Our multiresolution pyramid is built more efficiently and accurately than past work by decoupling pyramid filling from the measurement updates of different resolutions. To detect the saf…
Unmanned Aerial Vehicles (UAVs) have the potential to be used for many applications in urban environments. However, allowing UAVs to fly above densely populated areas raises concerns regarding safety. One of the main safety issues is the possibility for a failure to cause the loss of navigation capabilities, which can result in the UAV falling/landing in hazardous areas such as busy roads, where it can cause fatal accidents. Current standards, such as the SORA published in 2019, do not consider applicable mitigation techniques to handle this kind of hazardous situations. Consequently, certifyi…
Visual-based Safe Landing for UAVs in Populated Areas: Real-time Validation in Virtual Environments
Universidad Autónoma de Ciudad Juárez UACJ（华雷斯自治大学）；Center for Research in Mathematics CIMAT AC（数学研究中心CIMAT AC）；University of Texas at El Paso（德克萨斯大学埃尔帕索分校）
This paper addresses the problem of developing an algorithm for autonomous ship landing of vertical take-off and landing (VTOL) capable unmanned aerial vehicles (UAVs), using only a monocular camera in the UAV for tracking and localization. Ship landing is a challenging task due to the small landing space, six degrees of freedom ship deck motion, limited visual references for localization, and adversarial environmental conditions such as wind gusts. We first develop a computer vision algorithm which estimates the relative position of the UAV with respect to a horizon reference bar on the landi…
Evidence-Based Landing Site Selection and Vison-Based Landing for UAVs in Unstructured Environments
Aerospace Research Centre, National Research Council Canada（加拿大国家研究委员会航空航天研究中心）；Department of Mechanical and Industrial Engineering, Toronto Metropolitan University（多伦多 Metropolitan 大学机械与工业工程系）
AeroLite-MDNet: Lightweight Multi-task Deviation Detection Network for UAV Landing
Institute of Cyberspace Security, Zhejiang University of Technology（浙江工业大学网络安全学院）；College of Computer Science and Technology, Zhejiang University of Technology（浙江工业大学计算机科学与技术学院）；College of Geoinformatics, Zhejiang University of Technology（浙江工业大学地理信息学院）；Binjiang Institute of Artificial Intelligence, ZJUT（滨江人工智能研究院）；Quzhou Southeast Digital Economic Development Institute（庆州东南数字经济研究院）
Autonomous and safe landing is important for unmanned aerial vehicles. We present a monocular and stereo image based method for fast and accurate landing zone evaluation for UAVs in various scenarios. Many existing methods rely on Lidar or depth sensor to provide accurate and dense surface reconstruction. We utilize stereo images to evaluate the slope and monocular images to compute homography error. By combining them together, our approach works for both rigid and non-rigid dynamic surfaces. Experiments on many outdoor scenes such as water, grass and roofs, demonstrate the robustness and effe…
The paper discusses an intelligent vision-based control solution for autonomous tracking and landing of Vertical Take-Off and Landing (VTOL) capable Unmanned Aerial Vehicles (UAVs) on ships without utilizing GPS signal. The central idea involves automating the Navy helicopter ship landing procedure where the pilot utilizes the ship as the visual reference for long-range tracking; however, refers to a standardized visual cue installed on most Navy ships called the "horizon bar" for the final approach and landing phases. This idea is implemented using a uniquely designed nonlinear controller int…
A Manipulator-Assisted Multiple UAV Landing System for USV Subject to Disturbance
Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳市人工智能与机器人研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Maastricht University（马斯特里赫特大学）
This paper presents a system for hardware-in-the-loop (HiL) simulation of unmanned aerial vehicle (UAV) control algorithms implemented on a heterogeneous SoC FPGA computing platforms. The AirSim simulator running on a PC and an Arty Z7 development board with a Zynq SoC chip from AMD Xilinx were used. Communication was carried out via a serial USB link. An application for autonomous landing on a specially marked landing strip was selected as a case study. A landing site detection algorithm was implemented on the Zynq SoC platform. This allowed processing a 1280 x 720 @ 60 fps video stream in re…
Unmanned Aerial Vehicles grow more popular by the day and applications for them are crossing boundaries of science and industry, with everything from aerial photography to package delivery to disaster management benefiting from the technology. But before they become commonplace, there are challenges to be solved to make them reliable and safe. The following paper discusses the challenges associated with the precision landing of an Unmanned Aerial Vehicle, including methods for sensing and control and their merits and shortcomings for various applications.
Center for Research in Mathematics CIMAT AC（数学研究中心（CIMAT AC））
Autonomous landing of Unmanned Aerial Vehicles (UAVs) in crowded scenarios is crucial for successful deployment of UAVs in populated areas, particularly in emergency landing situations where the highest priority is to avoid hurting people. In this work, a new visual-based algorithm for identifying Safe Landing Zones (SLZ) in crowded scenarios is proposed, considering a camera mounted on an UAV, where the people in the scene move with unknown dynamics. To do so, a density map is generated for each image frame using a Deep Neural Network, from where a binary occupancy map is obtained aiming to o…
Soloviev Rybinsk State Aviation Technical University（索洛维约夫雷宾斯克国立航空技术大学）；NPP SATEK plus（NPP SATEK plus 公司）；Program System Institute of Russian Academy of Sciences（俄罗斯科学院程序系统研究所）；Institute For Systems Analysis of Russian Academy of Sciences（俄罗斯科学院系统分析研究所）
Autonomous fixed-wing UAV landing based on differential GPS is now a mainstream providing reliable and precise landing. But the task still remains challenging when GPS availability is limited like for military UAVs. We discuss a solution of this problem based on computer vision and dot markings along stationary or makeshift runway. We focus our attempts on using infrared beacons along with narrow-band filter as promising way to mark any makeshift runway and utilize particle filtering to fuse both IMU and visual data. We believe that unlike many other vision-based methods this solution is capab…
University of Petroleum and Energy Studies（石油与能源研究大学）；International Institute of Information Technology, Hyderabad（海得拉巴国际信息技术学院）
The world we live in is full of technology and with each passing day the advancement and usage of UAVs increases efficiently. As a result of the many application scenarios, there are some missions where the UAVs are vulnerable to external disruptions, such as a ground station's loss of connectivity, security missions, safety concerns, and delivery-related missions. Therefore, depending on the scenario, this could affect the operations and result in the safe landing of UAVs. Hence, this paper presents a heuristic approach towards safe landing of multi-rotor UAVs in the dynamic environments. The…
KIOS Research and Innovation Center of Excellence（KIOS卓越研究与创新中心）；University of Cyprus（塞浦路斯大学）
While Unmanned Aerial Vehicles (UAVs) are increasingly deployed in several missions, their inability of reliable and consistent autonomous landing poses a major setback for deploying such systems truly autonomously. In this paper we present an autonomous UAV landing system for landing on a moving platform. In contrast to existing attempts, the proposed system relies only on the camera sensor, and has been designed as lightweight as possible. The proposed system can be deployed on a low power platform as part of the drone payload, whilst being indifferent to any external communication or any ot…
Université de Toulouse（图卢兹大学）；LAAS-CNRS（国家科学研究中心自动化与系统分析实验室）；ONERA（法国国家航空航天研究院）
To certify UAV operations in populated areas, risk mitigation strategies -- such as Emergency Landing (EL) -- must be in place to account for potential failures. EL aims at reducing ground risk by finding safe landing areas using on-board sensors. The first contribution of this paper is to present a new EL approach, in line with safety requirements introduced in recent research. In particular, the proposed EL pipeline includes mechanisms to monitor learning based components during execution. This way, another contribution is to study the behavior of Machine Learning Runtime Monitoring (MLRM) a…
Safe autonomous landing in urban cities is a necessity for the growing Unmanned Aircraft Systems (UAS) industry. In urgent situations, building rooftops, particularly flat rooftops, can provide local safe landing zones for small UAS. This paper investigates the real-time identification and selection of safe landing zones on rooftops based on LiDAR and camera sensor feedback. A visual high fidelity simulated city is constructed in the Unreal game engine, with particular attention paid to accurately generating rooftops and the common obstructions found thereon, e.g., ac units, water towers, air …
Harnessing LSTM for Nonlinear Ship Deck Motion Prediction in UAV Autonomous Landing amidst High Sea States
Zhejiang Provincial Engineering Research Centre for Special Aircrafts（浙江省特种飞行器工程研究中心）；Ningbo Institute of Materials Technology & Engineering, CAS（中国科学院宁波材料技术与工程研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Faculty of Electrical Engineering and Computer Science, Ningbo University（宁波大学电气工程与计算机科学学院）
Unmanned Aerial Vehicles (UAVs) equipped with bioradars are a life-saving technology that can enable identification of survivors under collapsed buildings in the aftermath of natural disasters such as earthquakes or gas explosions. However, these UAVs have to be able to autonomously land on debris piles in order to accurately locate the survivors. This problem is extremely challenging as the structure of these debris piles is often unknown and no prior knowledge can be leveraged. In this work, we propose a computationally efficient system that is able to reliably identify safe landing sites an…
The vision of unmanned aerial vehicles is very significant for UAV-related applications such as search and rescue, landing on a moving platform, etc. In this work, we have developed an integrated system for the UAV landing on the moving platform, and the UAV object detection with tracking in the complicated environment. Firstly, we have proposed a robust LoG-based deep neural network for object detection and tracking, which has great advantages in robustness to object scale and illuminations compared with typical deep network-based approaches. Then, we have also improved based on the original …
IAT TU Darmstadt（达姆施塔特工业大学智能自动化研究所(IAT TU Darmstadt)）
In this paper, we introduce a dynamic fiducial marker which can change its appearance according to the spatiotemporal requirements of the visual perception task of a mobile robot using a camera as the sensor. We present a control scheme to dynamically change the appearance of the marker in order to increase the range of detection and to assure a better accuracy on the close range. The marker control takes into account the camera to marker distance (which influences the scale of the marker in image coordinates) to select which fiducial markers to display. Hence, we realize a tight coupling betw…
Risk Assessment for Autonomous Landing in Urban Environments using Semantic Segmentation
Center for Research in Mathematics, CIMAT AC（墨西哥数学研究中心（CIMAT AC））；Centro de Investigación en Matemáticas, A.C.（墨西哥数学研究中心）；Centro de Investigaciones en Óptica, A.C.（墨西哥光学研究中心）；CONAHCYT（墨西哥国家科学技术委员会）
As UAV technology improves, more uses have been found for these versatile autonomous vehicles, from surveillance to aerial photography, to package delivery, and each of these applications poses unique challenges. This paper implements a solution for one such challenge: To land on a moving target. This problem has been addressed before with varying degrees of success, however, most implementations focus on indoor applications. Outdoor poses greater challenges in the form of variables such as wind and lighting, and outdoor drones are heavier and more susceptible to inertial effects. Our approach…
SOAR: Regression-based LiDAR Relocalization for UAVs
Fujian Key Laboratory of Sensing and Computing for Smart Cities, Xiamen University（厦门大学智慧城市感知与计算重点实验室）；Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University（中国教育部多媒体可信感知与高效计算重点实验室）；Department of Computer Science at the University of Oxford（牛津大学计算机科学系）
A lot of UAV applications require vertical take-off and landing (VTOL) combined with very long-range or endurance. Transitioning UAVs have been proposed to combine the VTOL capabilities of helicopters with the efficient long-range flight properties of fixed-wing aircraft. But energy is still a bottleneck for many electric long endurance applications. While solar power technology and battery technology have improved a lot, in rougher conditions they still respectively lack the power or total amount of energy required for many real-world situations. In this paper, we introduce the NederDrone, a …
University of Hong Kong（香港大学）；Harbin Institute of Technology（哈尔滨工业大学）
The emergence of low-cost, small form factor and light-weight solid-state LiDAR sensors have brought new opportunities for autonomous unmanned aerial vehicles (UAVs) by advancing navigation safety and computation efficiency. Yet the successful developments of LiDAR-based UAVs must rely on extensive simulations. Existing simulators can hardly perform simulations of real-world environments due to the requirements of dense mesh maps that are difficult to obtain. In this paper, we develop a point-realistic simulator of real-world scenes for LiDAR-based UAVs. The key idea is the underlying point re…
Amazon（亚马逊公司）；West Virginia University（西弗吉尼亚大学）；Microsoft（微软公司）；Gather AI（Gather AI公司）；Mowito（Mowito公司）；Carnegie Mellon University（卡内基梅隆大学）
Many aerial robotic applications require the ability to land on moving platforms, such as delivery trucks and marine research boats. We present a method to autonomously land an Unmanned Aerial Vehicle on a moving vehicle. A visual servoing controller approaches the ground vehicle using velocity commands calculated directly in image space. The control laws generate velocity commands in all three dimensions, eliminating the need for a separate height controller. The method has shown the ability to approach and land on the moving deck in simulation, indoor and outdoor environments, and compared t…
Autonomous Systems Lab, ETH Zurich（苏黎世联邦理工学院自主系统实验室）；CSAIL, MIT（麻省理工学院计算机科学与人工智能实验室）
Full autonomy for fixed-wing unmanned aerial vehicles (UAVs) requires the capability to autonomously detect potential landing sites in unknown and unstructured terrain, allowing for self-governed mission completion or handling of emergency situations. In this work, we propose a perception system addressing this challenge by detecting landing sites based on their texture and geometric shape without using any prior knowledge about the environment. The proposed method considers hazards within the landing region such as terrain roughness and slope, surrounding obstacles that obscure the landing ap…
Fusion of Visual-Inertial Odometry with LiDAR Relative Localization for Cooperative Guidance of a Micro-Scale Aerial Vehicle
Multi-robot Systems Group, Department of Cybernetics, Faculty of Electrical Engineering, Czech Technical University in Prague, Technick\'a 2, Prague, 166 27, Czech Republic
An Experimental Study of Trojan Vulnerabilities in UAV Autonomous Landing
Department of Electrical and Computer Engineering at North Carolina A&T State University（北卡罗来纳A&T州立大学电气与计算机工程系）；University of Alabama（阿拉巴马大学）；Department of Electrical and Computer Engineering, Windsor University（温莎大学电气与计算机工程系）
With the development of industry, drones are appearing in various field. In recent years, deep reinforcement learning has made impressive gains in games, and we are committed to applying deep reinforcement learning algorithms to the field of robotics, moving reinforcement learning algorithms from game scenarios to real-world application scenarios. We are inspired by the LunarLander of OpenAI Gym, we decided to make a bold attempt in the field of reinforcement learning to control drones. At present, there is still a lack of work applying reinforcement learning algorithms to robot control, the p…
Faculty of Electrical Engineering, Czech Technical University in Prague（捷克技术大学电气工程学院）
Autonomous operation of UAVs in a closed environment requires precise and reliable pose estimate that can stabilize the UAV without using external localization systems such as GNSS. In this work, we are concerned with estimating the pose from laser scans generated by an inexpensive and lightweight LIDAR. We propose a localization system for lightweight (under 200g) LIDAR sensors with high reliability in arbitrary environments, where other methods fail. The general nature of the proposed method allows deployment in wide array of applications. Moreover, seamless transitioning between different k…
