Multirotor UAVs are used for a wide spectrum of civilian and public domain applications. Navigation controllers endowed with different attributes and onboard sensor suites enable multirotor autonomous or semi-autonomous, safe flight, operation, and functionality under nominal and detrimental conditions and external disturbances, even when flying in uncertain and dynamically changing environments. During the last decade, given the faster-than-exponential increase of available computational power, different learning-based algorithms have been derived, implemented, and tested to navigate and control, among other systems, multirotor UAVs. Learning algorithms have been, and are used to derive data-driven based models, to identify parameters, to track objects, to develop navigation controllers, and to learn the environment in which multirotors operate. Learning algorithms combined with model-based control techniques have been proven beneficial when applied to multirotors. This survey summarizes published research since 2015, dividing algorithms, techniques, and methodologies into offline and online learning categories, and then, further classifying them into machine learning, deep learning, and reinforcement learning sub-categories. An integral part and focus of this survey are on online learning algorithms as applied to multirotors with the aim to register the type of learning techniques that are either hard or almost hard real-time implementable, as well as to understand what information is learned, why, and how, and how fast. The outcome of the survey offers a clear understanding of the recent state-of-the-art and of the type and kind of learning-based algorithms that may be implemented, tested, and executed in real-time.
The demand for accurate and fast trajectory tracking for multirotor Unmanned Aerial Vehicles (UAVs) have grown recently due to advances in UAV avionics technology and application domains. In many applications, the multirotor UAV is required to accurately perform aggressive maneuvers in challenging scenarios like the presence of external wind disturbances or in-flight payload changes. In this paper, we propose a systematic controller tuning approach based on identification results obtained by a recently developed Deep Neural Networks with the Modified Relay Feedback Test (DNN-MRFT) algorithm. W…
This paper presents a new method for integrated time-optimal routing and trajectory optimization of multirotor unmanned aerial vehicles (UAVs). Our approach extends the well-known Traveling Salesman Problem by accounting for the limited maneuverability of the UAVs due to their kinematic properties. To this end, we allow each waypoint to be traversed with a discretized velocity as well as a discretized flight direction and compute time-optimal trajectories to determine the travel time costs for each edge. We refer to this novel optimization problem as the Trajectory-based Traveling Salesman Pro…
Defence Research and Development Canada（加拿大国防研究与发展部）；University of Toronto Institute for Aerospace Studies (UTIAS), University of Toronto（多伦多大学多伦多航空航天研究所（UTIAS））
Redundant navigation systems are critical for safe operation of UAVs in high-risk environments. Since most commercial UAVs almost wholly rely on GPS, jamming, interference and multi-pathing are real concerns that usually limit their operations to low-risk environments and Visual Line-Of-Sight. This paper presents a vision-based route-following system for the autonomous, safe return of UAVs under primary navigation failure such as GPS jamming. Using a Visual Teach & Repeat framework to build a visual map of the environment during an outbound flight, we show the autonomous return of the UAV by v…
In many unmanned aerial vehicle (UAV) applications for surveillance and data collection, it is not possible to reach all requested locations due to the given maximum flight time. Hence, the requested locations must be prioritized and the problem of selecting the most important locations is modeled as an Orienteering Problem (OP). To fully exploit the kinematic properties of the UAV in such scenarios, we combine the OP with the generation of time-optimal trajectories with bounds on velocity and acceleration. We define the resulting problem as the Kinematic Orienteering Problem (KOP) and propose…
Multi-Domain Physics-Based MDO of Multirotor UAVs: A Deterministic Framework for Discrete COTS Sizing
International Institute of Information Technology, Hyderabad（印度海得拉巴国际信息技术学院）；Indian Institute of Science Education and Research, Bhopal（印度博帕尔科学教育与研究学院）
Unmanned aerial vehicles (UAVs) are finding use in applications that place increasing emphasis on robustness to external disturbances including extreme wind. However, traditional multirotor UAV platforms do not directly sense wind; conventional flow sensors are too slow, insensitive, or bulky for widespread integration on UAVs. Instead, drones typically observe the effects of wind indirectly through accumulated errors in position or trajectory tracking. In this work, we integrate a novel flow sensor based on micro-electro-mechanical systems (MEMS) hot-wire technology developed in our prior wor…
Energy management is a critical aspect of risk assessment for Uncrewed Aerial Vehicle (UAV) flights, as a depleted battery during a flight brings almost guaranteed vehicle damage and a high risk of human injuries or property damage. Predicting the amount of energy a flight will consume is challenging as routing, weather, obstacles, and other factors affect the overall consumption. We develop a deep energy model for a UAV that uses Temporal Convolutional Networks to capture the time varying features while incorporating static contextual information. Our energy model is trained on a real world d…
AirBender: Adaptive Transportation of Bendable Objects Using Dual UAVs
Autonomous and Intelligent Robotics Laboratory (AIRLab)（自主与智能机器人实验室）；Lehigh University（莱斯大学）；Electrical and Computer Engineering Department（电子与计算机工程系）；The University of New Mexico（新墨西哥大学）
Recently, needs for unmanned aerial vehicles (UAVs) that are attachable to the wall have been highlighted. As one of the ways to address the need, researches on various tilting multirotors that can increase maneuverability has been employed. Unfortunately, existing studies on the tilting multirotors require considerable amounts of prior information on the complex dynamic model. Meanwhile, reinforcement learning on quadrotors has been studied to mitigate this issue. Yet, these are only been applied to standard quadrotors, whose systems are less complex than those of tilting multirotors. In this…
Force Polytope-Based Cant-Angle Selection for Tilting Hexarotor UAVs
University of Padova（帕多瓦大学）；Department of Management and Engineering（管理与工程系）；Department of Information Engineering（信息工程系）；Department of Industrial Engineering（工业工程系）
The field of Unmanned Aerial Vehicles (UAVs) has reached a high level of maturity in the last few years. Hence, bringing such platforms from closed labs, to day-to-day interactions with humans is important for commercialization of UAVs. One particular human-UAV scenario of interest for this paper is the payload handover scheme, where a UAV hands over a payload to a human upon their request. In this scope, this paper presents a novel real-time human-UAV interaction detection approach, where Long short-term memory (LSTM) based neural network is developed to detect state profiles resulting from h…
Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院）；Tsinghua University（清华大学）；Peking University（北京大学）
Tracking position and orientation independently affords more agile maneuver for over-actuated multirotor Unmanned Aerial Vehicles (UAVs) while introducing undesired downwash effects; downwash flows generated by thrust generators may counteract others due to close proximity, which significantly threatens the stability of the platform. The complexity of modeling aerodynamic airflow challenges control algorithms from properly compensating for such a side effect. Leveraging the input redundancies in over-actuated UAVs, we tackle this issue with a novel control allocation framework that considers d…
King Abdullah University of Science & Technology（阿卜杜拉国王科技大学）；University of Maryland（马里兰大学）；Polytechnique Montreal（蒙特利尔理工学院）；Australian College of Kuwait（科威特澳大利亚学院）
We present the Robotics Intelligent Systems & Control (RISC) Lab multiagent testbed for reliable search and rescue and aerial transport in outdoor environments. The system consists of a team of three multirotor unmanned aerial vehicles (UAVs), which are capable of autonomously searching, picking up, and transporting randomly distributed objects in an outdoor field. The method involves vision based object detection and localization, passive aerial grasping with our novel design, GPS based UAV navigation, and safe release of the objects at the drop zone. Our cooperative strategy ensures safe spa…
This paper presents PANTHER, a real-time perception-aware (PA) trajectory planner for multirotor-UAVs (Unmanned Aerial Vehicles) in dynamic environments. PANTHER plans trajectories that avoid dynamic obstacles while also keeping them in the sensor field of view (FOV) and minimizing the blur to aid in object tracking. The rotation and translation of the UAV are jointly optimized, which allows PANTHER to fully exploit the differential flatness of multirotors to maximize the PA objective. Real-time performance is achieved by implicitly imposing the underactuated dynamics of the UAV through the Ho…
In this paper we propose an effective vision-based navigation method that allows a multirotor vehicle to simultaneously reach a desired goal pose in the environment while constantly facing a target object or landmark. Standard techniques such as Position-Based Visual Servoing (PBVS) and Image-Based Visual Servoing (IBVS) in some cases (e.g., while the multirotor is performing fast maneuvers) do not allow to constantly maintain the line of sight with a target of interest. Instead, we compute the optimal trajectory by solving a non-linear optimization problem that minimizes the target re-project…
Indian Institute of Technology Kharagpur（印度理工学院卡拉格普尔分校）；University of Stuttgart（斯图加特大学）
UAVs have found an important application in archaeological mapping. Majority of the existing methods employ an offline method to process the data collected from an archaeological site. They are time-consuming and computationally expensive. In this paper, we present a multi-UAV approach for faster mapping of archaeological sites. Employing a team of UAVs not only reduces the mapping time by distribution of coverage area, but also improves the map accuracy by exchange of information. Through extensive experiments in a realistic simulation (AirSim), we demonstrate the advantages of using a collab…
In recent years, the proliferation of unmanned aerial vehicles (UAVs) has increased dramatically. UAVs can accomplish complex or dangerous tasks in a reliable and cost-effective way but are still limited by power consumption problems, which pose serious constraints on the flight duration and completion of energy-demanding tasks. The possibility of providing UAVs with advanced decision-making capabilities in an energy-effective way would be extremely beneficial. In this paper, we propose a practical solution to this problem that exploits deep learning on the edge. The developed system integrate…
RELAX: Reinforcement Learning Enabled 2D-LiDAR Autonomous System for Parsimonious UAVs
School of Data Science, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）数据科学学院）；University of Chicago（芝加哥大学）；University of California, Los Angeles（加州大学洛杉矶分校）
Neural 3D Object Reconstruction with Small-Scale Unmanned Aerial Vehicles
i2CAT Foundation（i2CAT基金会）；University of Klagenfurt（克雷格弗特大学）；Universitat Politècnica de Catalunya（巴塞罗那理工大学）；Faculty of Electrical Engineering and Computing（电气工程与计算学院）；University of Zagreb（扎格雷布大学）；NEC Labs Europe GmbH（NEC欧洲实验室）；ICREA
In this paper, we introduce an informative path planning (IPP) framework for active classification using unmanned aerial vehicles (UAVs). Our algorithm uses a combination of global viewpoint selection and evolutionary optimization to refine the planned trajectory in continuous 3D space while satisfying dynamic constraints. Our approach is evaluated on the application of weed detection for precision agriculture. We model the presence of weeds on farmland using an occupancy grid and generate adaptive plans according to information-theoretic objectives, enabling the UAV to gather data efficiently…
The capabilities of autonomous flight with unmanned aerial vehicles (UAVs) have significantly increased in recent times. However, basic problems such as fast and robust geo-localization in GPS-denied environments still remain unsolved. Existing research has primarily concentrated on improving the accuracy of localization at the cost of long and varying computation time in various situations, which often necessitates the use of powerful ground station machines. In order to make image-based geo-localization online and pragmatic for lightweight embedded systems on UAVs, we propose a framework tha…
Occupancy grid maps (OGMs) are fundamental to most systems for autonomous robotic navigation. However, CPU-based implementations struggle to keep up with data rates from modern 3D lidar sensors, and provide little capacity for modern extensions which maintain richer voxel representations. This paper presents OHM, our open source, GPU-based OGM framework. We show how the algorithms can be mapped to GPU resources, resolving difficulties with contention to obtain a successful implementation. The implementation supports many modern OGM algorithms including NDT-OM, NDT-TM, decay-rate and TSDF. A th…
Spatial-Semantic Reasoning using Large Language Models for Efficient UAV Search Operations
University of Zagreb（萨格勒布大学）；Faculty of Electrical Engineering and Computing（电气工程与计算机学院）；LARICS (Laboratory for Robotics and Intelligent Control Systems)（拉里克机器人与智能控制系统实验室）
RALLY: Role-Adaptive LLM-Driven Yoked Navigation for Agentic UAV Swarms
Hangzhou Institute for Advanced Study, UCAS（杭州高等研究机构，中国科学院大学）；Zhejiang University（浙江大学）；Zhejiang Lab（浙江实验室）；Macau University of Science and Technology（澳门科学理工学院）
National Robotics Engineering Center, the Robotics Institute of Carnegie Mellon University（卡内基梅隆大学机器人研究所国家机器人工程中心）
A robotic system of multiple unmanned ground vehicles (UGVs) and unmanned aerial vehicles (UAVs) has the potential for advancing autonomous object geolocation performance. Much research has focused on algorithmic improvements on individual components, such as navigation, motion planning, and perception. In this paper, we present a UGV-UAV object detection and geolocation system, which performs perception, navigation, and planning autonomously in real scale in unstructured environment. We designed novel sensor pods equipped with multispectral (visible, near-infrared, thermal), high resolution (…
