Morphable design and depth-based visual control are two upcoming trends leading to advancements in the field of quadrotor autonomy. Stereo-cameras have struck the perfect balance of weight and accuracy of depth estimation but suffer from the problem of depth range being limited and dictated by the baseline chosen at design time. In this paper, we present a framework for quadrotor navigation based on a stereo camera system whose baseline can be adapted on-the-fly. We present a method to calibrate the system at a small number of discrete baselines and interpolate the parameters for the entire baseline range. We present an extensive theoretical analysis of calibration and synchronization errors. We showcase three different applications of such a system for quadrotor navigation: (a) flying through a forest, (b) flying through an unknown shaped/location static/dynamic gap, and (c) accurate 3D pose detection of an independently moving object. We show that our variable baseline system is more accurate and robust in all three scenarios. To our knowledge, this is the first work that applies the concept of morphable design to achieve a variable baseline stereo vision system on a quadrotor.
University of Michigan（密歇根大学）；University of Arizona（亚利桑那大学）
This paper develops and experimentally evaluates a navigation function for quadrotor formation flight that is resilient to abrupt quadrotor failures and other obstacles. The navigation function is based on modeling healthy quadrotors as particles in an ideal fluid flow. We provide three key contributions: (i) A Containment Exclusion Mode (CEM) safety theorem and proof which guarantees safety and formally specifies a minimum safe distance between quadrotors in formation, (ii) A real-time, computationally efficient CEM navigation algorithm, (iii) Simulation and experimental algorithm validation.…
Institute for Systems Analysis of Russian Academy of Sciences（俄罗斯科学院系统分析研究所）；Soloviev Rybinsk State Aviation Technical University（索洛维约夫雷宾斯克国立航空技术大学）；NPP SATEK plus（SATEK plus公司）
The problem of developing distributed control and navigation system for quadrotor UAVs operating in GPS-denied environments is addressed in the paper. Cooperative navigation, marker detection and mapping task solved by a team of multiple unmanned aerial vehicles is chosen as demo example. Developed intelligent control system complies with on 4D\RCS reference model and its implementation is based on ROS framework. Custom implementation of EKF-based map building algorithm is used to solve marker detection and map building task.
Recently, Model Predictive Path Integral (MPPI) control algorithm has been extensively applied to autonomous navigation tasks, where the cost map is mostly assumed to be known and the 2D navigation tasks are only performed. In this paper, we propose a generic MPPI control framework that can be used for 2D or 3D autonomous navigation tasks in either fully or partially observable environments, which are the most prevalent in robotics applications. This framework exploits directly the 3D-voxel grid acquired from an on-board sensing system for performing collision-free navigation. We test the fram…
In this work we showcase the design and assessment of the performance of a multi-camera UAV, when coupled with state-of-the-art planning and mapping algorithms for autonomous navigation. The system leverages state-of-the-art receding horizon exploration techniques for Next-Best-View (NBV) planning with 3D and semantic information, provided by a reconfigurable multi stereo camera system. We employ our approaches in an autonomous drone-based inspection task and evaluate them in an autonomous exploration and mapping scenario. We discuss the advantages and limitations of using multi stereo camera …
KTH Royal Institute of Technology（瑞典皇家理工学院）；Cyprus University of Technology（塞浦路斯理工大学）；University of Cyprus（塞浦路斯大学）；Aalto University（阿尔托大学）
We propose a multisensor fusion framework for onboard real-time navigation of a quadrotor in an indoor environment, by integrating sensor readings from an Inertial Measurement Unit (IMU), a camera-based object detection algorithm, and an Ultra-WideBand (UWB) localization system. The sensor readings from the camera-based object detection algorithm and the UWB localization system arrive intermittently, since the measurements are not readily available. We design a Kalman filter that manages intermittent observations in order to handle and fuse the readings and estimate the pose of the quadrotor f…
We present a novel stereo vision algorithm that is capable of obstacle detection on a mobile-CPU processor at 120 frames per second. Our system performs a subset of standard block-matching stereo processing, searching only for obstacles at a single depth. By using an onboard IMU and state-estimator, we can recover the position of obstacles at all other depths, building and updating a full depth-map at framerate. Here, we describe both the algorithm and our implementation on a high-speed, small UAV, flying at over 20 MPH (9 m/s) close to obstacles. The system requires no external sensing or com…
School of Electronics and Communication Enginnering, Sun Yet-sen University, Shenzhen, China（电子与通信工程学院，孙逸森大学，深圳，中国）；Tianwei Xunda (Hunan) Technology Co., Ltd., Hunan, China（天威湘电（湖南）科技有限公司，湖南，中国）
Visual navigation algorithms for quadrotors often exhibit a large variation in performance when transferred across different vehicle platforms and scene geometries, which increases the cost and risk of field deployment. To support systematic early-stage evaluation, we introduce FLYINGTRUST, a high-fidelity, configurable benchmarking framework that measures how platform kinodynamics and scenario structure jointly affect navigation robustness. FLYINGTRUST models vehicle capability with two compact, physically interpretable indicators: maximum thrust-to-weight ratio and axis-wise maximum angular …
High speed navigation through unknown environments is a challenging problem in robotics. It requires fast computation and tight integration of all the subsystems on the robot such that the latency in the perception-action loop is as small as possible. Aerial robots add a limitation of payload capacity, which restricts the amount of computation that can be carried onboard. This requires efficient algorithms for each component in the navigation system. In this paper, we describe our quadrotor system which is able to smoothly navigate through mixed indoor and outdoor environments and is able to f…
Geometry-morphing Micro Aerial Vehicles (MAVs) are gaining more and more attention lately, since their ability to modify their geometric morphology in-flight increases their versatility, while expanding their application range. In this novel research field, most of the works focus on the platform design and on the low-level control part for maintaining stability after the deformation. Nevertheless, another aspect of geometry morphing MAVs is the association of the deformation with respect to the shape and structure of the environment. In this article, we propose a novel Nonlinear Model Predict…
The increasing popularity of quadrotors has given rise to a class of predominantly vision-driven vehicles. This paper addresses the problem of perception-aware time optimal path parametrization for quadrotors. Although many different choices of perceptual modalities are available, the low weight and power budgets of quadrotor systems makes a camera ideal for on-board navigation and estimation algorithms. However, this does come with a set of challenges. The limited field of view of the camera can restrict the visibility of salient regions in the environment, which dictates the necessity to con…
KTH Royal Institute of Technology（KTH皇家理工学院）；University of Cyprus（塞浦路斯大学）；Aalto University（阿尔托大学）；Cyprus University of Technology（塞浦路斯理工大学）
We present a multisensor fusion framework for the onboard real-time navigation of a quadrotor in an indoor environment. The framework integrates sensor readings from an Inertial Measurement Unit (IMU), a camera-based object detection algorithm, and an Ultra-WideBand (UWB) localisation system. Often the sensor readings are not always readily available, leading to inaccurate pose estimation and hence poor navigation performance. To effectively handle and fuse sensor readings, and accurately estimate the pose of the quadrotor for tracking a predefined trajectory, we design a Maximum Correntropy C…
This paper addresses the problem of real-time vision-based autonomous obstacle avoidance in unstructured environments for quadrotor UAVs. We assume that our UAV is equipped with a forward facing stereo camera as the only sensor to perceive the world around it. Moreover, all the computations are performed onboard. Feasible trajectory generation in this kind of problems requires rapid collision checks along with efficient planning algorithms. We propose a trajectory generation approach in the depth image space, which refers to the environment information as depicted by the depth images. In order…
Aerial Inspection Behaviors via RL-based Quadrotor Control for Under-canopy Forest Environments
Robotics and AI group, in the Department of Computer Science, Electrical and Space Engineering at Luleå University of Technology, Sweden（鲁尔坎大学技术学院机器人与人工智能小组，计算机科学、电气与空间工程系，瑞典）
SwordRiding: A Unified Navigation Framework for Quadrotors in Unknown Complex Environments via Online Guiding Vector Fields
Pengcheng Laboratory, Shenzhen, Guangdong, China（鹏城实验室，深圳，广东，中国）；School of Automation, Beijing Institute of Technology, Beijing, China（自动化学院，北京理工大学，北京，中国）；School of Artificial Intelligence and Robotics, Hunan University（人工智能与机器人学院，湖南大学）；Department of Control Science and Engineering, Harbin Institute of Technology（控制科学与工程学院，哈尔滨工业大学）；National Key Laboratory of Autonomous Intelligent Unmanned Systems（自主智能无人系统国家重点实验室）；Department of Mechanical and Automation Engineering, the Chinese University of Hong Kong, Hong Kong, China（机械与自动化工程系，香港中文大学，香港，中国）
Hong Kong University of Science and Technology（香港科技大学）；Zhejiang University（浙江大学）
Recent advances in trajectory replanning have enabled quadrotor to navigate autonomously in unknown environments. However, high-speed navigation still remains a significant challenge. Given very limited time, existing methods have no strong guarantee on the feasibility or quality of the solutions. Moreover, most methods do not consider environment perception, which is the key bottleneck to fast flight. In this paper, we present RAPTOR, a robust and perception-aware replanning framework to support fast and safe flight. A path-guided optimization (PGO) approach that incorporates multiple topolog…
In this work, we address the problem of computation time for trajectory generation in quadrotors. Most trajectory generation methods for waypoint navigation of quadrotors, for example minimum snap/jerk and minimum-time, are structured as bi-level optimizations. The first level involves allocating time across all input waypoints and the second step is to minimize the snap/jerk of the trajectory under that time allocation. Such an optimization can be computationally expensive to solve. In our approach we treat trajectory generation as a supervised learning problem between a sequential set of inp…
Despite advances in localization and navigation, aerial robots inevitably remain susceptible to accidents and collisions. In this work, we propose a passive foldable airframe as a protective mechanism for a small aerial robot. A foldable quadrotor is designed and fabricated using the origami-inspired manufacturing paradigm. Upon an accidental mid-flight collision, the deformable airframe is mechanically activated. The rigid frame reconfigures its structure to protect the central part of the robot that houses sensitive components from a crash to the ground. The proposed robot is fabricated, mod…
We study how autonomous robots can learn by themselves to improve their depth estimation capability. In particular, we investigate a self-supervised learning setup in which stereo vision depth estimates serve as targets for a convolutional neural network (CNN) that transforms a single still image to a dense depth map. After training, the stereo and mono estimates are fused with a novel fusion method that preserves high confidence stereo estimates, while leveraging the CNN estimates in the low-confidence regions. The main contribution of the article is that it is shown that the fused estimates …
Minimum-time navigation within constrained and dynamic environments is of special relevance in robotics. Seeking time-optimality, while guaranteeing the integrity of time-varying spatial bounds, is an appealing trade-off for agile vehicles, such as quadrotors. State of the art approaches, either assume bounds to be static and generate time-optimal trajectories offline, or compromise time-optimality for constraint satisfaction. Leveraging nonlinear model predictive control and a path parametric reformulation of the quadrotor model, we present a real-time control that approximates time-optimal b…
MAD: Mapping-Aware World Models for Agile Quadrotor Flight
State Key Lab of Autonomous Intelligent Unmanned Systems, Beijing Institute of Technology, Beijing 100081, China（自主智能无人系统国家重点实验室，北京理工大学，北京100081，中国）；Zhongguancun Academy, Beijing 100094, China（中关村学院，北京100094，中国）；School of Computer Science and Technology, Tongji University（同济大学计算机科学与技术学院）；Department of Mechanical and Energy Engineering, Southern University of Science and Technology（南方科技大学机械与能源工程系）；Harbin Institute of Technology（哈尔滨工业大学）
Roller-Quadrotor: A Novel Hybrid Terrestrial/Aerial Quadrotor with Unicycle-Driven and Rotor-Assisted Turning
Zhejiang University（浙江大学）；School of Mechanical Engineering, Zhejiang University（浙江大学机械工程学院）；Robotics Institute of Zhejiang University（浙江大学机器人研究所）；Robotics Research Center of Yuyao City（余姚市机器人研究中心）；The State Key Laboratory of Industrial Control Technology, Zhejiang University（浙江大学工业控制技术国家重点实验室）；Huzhou Institute of Zhejiang University（浙江大学湖州研究院）
University of Maryland Institute for Advanced Computer Studies（马里兰大学高级计算机研究所）
Although quadrotors, and aerial robots in general, are inherently active agents, their perceptual capabilities in literature so far have been mostly passive in nature. Researchers and practitioners today use traditional computer vision algorithms with the aim of building a representation of general applicability: a 3D reconstruction of the scene. Using this representation, planning tasks are constructed and accomplished to allow the quadrotor to demonstrate autonomous behavior. These methods are inefficient as they are not task driven and such methodologies are not utilized by flying insects a…
East China University of Science and Technology（华东理工大学）；Western Sydney University（西悉尼大学）
The ability to perform aggressive movements, which are called aggressive flights, is important for quadrotors during navigation. However, aggressive quadrotor flights are still a great challenge to practical applications. The existing solutions to aggressive flights heavily rely on a predefined trajectory, which is a time-consuming preprocessing step. To avoid such path planning, we propose a curiosity-driven reinforcement learning method for aggressive flight missions and a similarity-based curiosity module is introduced to speed up the training procedure. A branch structure exploration (BSE)…
NVIDIA（英伟达）；University of California, Santa Barbara（加州大学圣巴巴拉分校）
Stereo-based depth estimation is a cornerstone of computer vision, with state-of-the-art methods delivering accurate results in real time. For several applications such as autonomous navigation, however, it may be useful to trade accuracy for lower latency. We present Bi3D, a method that estimates depth via a series of binary classifications. Rather than testing if objects are at a particular depth $D$, as existing stereo methods do, it classifies them as being closer or farther than $D$. This property offers a powerful mechanism to balance accuracy and latency. Given a strict time budget, Bi3…
Augmented reality (AR) technology has been introduced into the robotics field to narrow the visual gap between indoor and outdoor environments. However, without signals from satellite navigation systems, flight experiments in these indoor AR scenarios need other accurate localization approaches. This work proposes a real-time centimeter-level indoor localization method based on psycho-visually invisible projected tags (IPT), requiring a projector as the sender and quadrotors with high-speed cameras as the receiver. The method includes a modulation process for the sender, as well as demodulatio…
Rotor-Failure-Aware Quadrotors Flight in Unknown Environments
School of Robotics and Automation, Nanjing University（南京大学机器人与自动化学院）；Institute of Cyber-Systems and Control, College of Control Science and Engineering, Zhejiang University（浙江大学控制系统研究所）；Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University（香港理工大学航空工程系）；School of Aeronautic Science and Engineering, Beihang University（北航航空科学与工程学院）
Perception-aware Planning for Quadrotor Flight in Unknown and Feature-limited Environments
School of Intelligence Science and Engineering, Harbin Institute of Technology（哈尔滨工业大学智能科学与工程学院）；Department of Mechanical and Energy Engineering, Southern University of Science and Technology（南方科技大学机械与能源工程系）
City University of Hong Kong（香港城市大学）；The Chinese University of Hong Kong（香港中文大学）；Soochow University（苏州大学）
Objective: The computation of anatomical information and laparoscope position is a fundamental block of surgical navigation in Minimally Invasive Surgery (MIS). Recovering a dense 3D structure of surgical scene using visual cues remains a challenge, and the online laparoscopic tracking primarily relies on external sensors, which increases system complexity. Methods: Here, we propose a learning-driven framework, in which an image-guided laparoscopic localization with 3D reconstructions of complex anatomical structures is obtained. To reconstruct the 3D structure of the whole surgical environmen…
Zhejiang University（浙江大学）；National Engineering Research Center for Industrial Automation (Ningbo Institute)（国家工业自动化工程技术研究中心（宁波研究院））；Huzhou Institute, Zhejiang University（浙江大学湖州研究院）
This paper presents a decentralized and asynchronous systematic solution for multi-robot autonomous navigation in unknown obstacle-rich scenes using merely onboard resources. The planning system is formulated under gradient-based local planning framework, where collision avoidance is achieved by formulating the collision risk as a penalty of a nonlinear optimization problem. In order to improve robustness and escape local minima, we incorporate a lightweight topological trajectory generation method. Then agents generate safe, smooth, and dynamically feasible trajectories in only several millis…
University of Maryland（马里兰大学）；University of Zurich（苏黎世大学）；ETH Zurich（苏黎世联邦理工学院）
Dynamic obstacle avoidance on quadrotors requires low latency. A class of sensors that are particularly suitable for such scenarios are event cameras. In this paper, we present a deep learning -- based solution for dodging multiple dynamic obstacles on a quadrotor with a single event camera and on-board computation. Our approach uses a series of shallow neural networks for estimating both the ego-motion and the motion of independently moving objects. The networks are trained in simulation and directly transfer to the real world without any fine-tuning or retraining. We successfully evaluate an…
Dense 3D maps from wide-angle cameras is beneficial to robotics applications such as navigation and autonomous driving. In this work, we propose a real-time dense 3D mapping method for fisheye cameras without explicit rectification and undistortion. We extend the conventional variational stereo method by constraining the correspondence search along the epipolar curve using a trajectory field induced by camera motion. We also propose a fast way of generating the trajectory field without increasing the processing time compared to conventional rectified methods. With our implementation, we were a…
Zhejiang University（浙江大学）；Zhejiang University of Technology（浙江工业大学）；Northwestern Polytechnical University（西北工业大学）
Autonomous drone racing is becoming an excellent platform to challenge quadrotors' autonomy techniques including planning, navigation and control technologies. However, most research on this topic mainly focuses on single drone scenarios. In this paper, we describe a novel time-optimal trajectory generation method for generating time-optimal trajectories for a swarm of quadrotors to fly through pre-defined waypoints with their maximum maneuverability without collision. We verify the method in the Gazebo simulations where a swarm of 5 quadrotors can fly through a complex 6-waypoint racing track…
