In many scenarios, unmanned aerial vehicles (UAVs), aka drones, need to have the capability of autonomous flying to carry out their mission successfully. In order to allow these autonomous flights, drones need to know their location constantly. Then, based on the current position and the final destination, navigation commands will be generated and drones will be guided to their destination. Localization can be easily carried out in outdoor environments using GPS signals and drone inertial measurement units (IMUs). However, such an approach is not feasible in indoor environments or GPS-denied areas. In this paper, we propose a localization scheme for drones called PILOT (High-Precision Indoor Localization for Autonomous Drones) that is specifically designed for indoor environments. PILOT relies on ultrasonic acoustic signals to estimate the target drone's location. In order to have a precise final estimation of the drone's location, PILOT deploys a three-stage localization scheme. The first two stages provide robustness against the multi-path fading effect of indoor environments and mitigate the ranging error. Then, in the third stage, PILOT deploys a simple yet effective technique to reduce the localization error induced by the relative geometry between transmitters and receivers and significantly reduces the height estimation error. The performance of PILOT was assessed under different scenarios and the results indicate that PILOT achieves centimeter-level accuracy for three-dimensional localization of drones.
For many applications, drones are required to operate entirely or partially autonomously. To fly completely or partially on their own, drones need access to location services to get navigation commands. While using the Global Positioning System (GPS) is an obvious choice, GPS is not always available, can be spoofed or jammed, and is highly error-prone for indoor and underground environments. The ranging method using beacons is one of the popular methods for localization, specially for indoor environments. In general, localization error in this class is due to two factors: the ranging error and…
As radar sensors are being miniaturized, there is a growing interest for using them in indoor sensing applications such as indoor drone obstacle avoidance. In those novel scenarios, radars must perform well in dense scenes with a large number of neighboring scatterers. Central to radar performance is the detection algorithm used to separate targets from the background noise and clutter. Traditionally, most radar systems use conventional CFAR detectors but their performance degrades in indoor scenarios with many reflectors. Inspired by the advances in non-linear target detection, we propose a n…
KU Leuven（鲁汶大学）；Ghent University（根特大学）；imec（比利时微电子研究中心）
Frequency-modulated continuous-wave (FMCW) radar is a promising sensor technology for indoor drones as it provides range, angular as well as Doppler-velocity information about obstacles in the environment. Recently, deep learning approaches have been proposed for processing FMCW data, outperforming traditional detection techniques on range-Doppler or range-azimuth maps. However, these techniques come at a cost; for each novel task a deep neural network architecture has to be trained on high-dimensional input data, stressing both data bandwidth and processing budget. In this paper, we investiga…
Humans race drones faster than algorithms, despite being limited to a fixed camera angle, body rate control, and response latencies in the order of hundreds of milliseconds. A better understanding of the ability of human pilots of selecting appropriate motor commands from highly dynamic visual information may provide key insights for solving current challenges in vision-based autonomous navigation. This paper investigates the relationship between human eye movements, control behavior, and flight performance in a drone racing task. We collected a multimodal dataset from 21 experienced drone pil…
First-person view drone racing has become a popular televised sport. However, very little is known about the perceptual and motor skills of professional drone racing pilots. A better understanding of these skills may inform path planning and control algorithms for autonomous multirotor flight. By using a real-world drone racing track and a large-scale position tracking system, we compare the drone racing performance of five professional and five beginner pilots. Results show that professional pilots consistently outperform beginner pilots by achieving faster lap times, higher velocity, and mor…
Universidad Tecnológica del Uruguay（乌拉圭技术大学）；Escuela Técnica Superior de Rivera（里韦拉高等技术学校）
Commercial drones are not yet dimensioned to perform indoor autonomous tasks, since they use GPS for their location in the environment. When it comes to a space with physical obstacles (walls, metal, etc.) between the communication of the drone and the satellites that allow the precise location of the same, there is great difficulty in finding the satellites or it generates interference for this location. This problem can cause an unexpected action of the drone, a collision and a possible accident can occur. The work to follow presents the development of a drone capable of operating in a physi…
ETH Zürich（苏黎世联邦理工学院）；inspire AG（Inspire股份公司）；Tinamu Labs（蒂纳穆实验室）
Volume estimation in large indoor spaces is an important challenge in robotic inspection of industrial warehouses. We propose an approach for volume estimation for autonomous systems using visual features for indoor localization and surface reconstruction from 2D-LiDAR measurements. A Gaussian Process-based model incorporates information collected from measurements given statistical prior information about the terrain, from which the volume estimate is computed. Our algorithm finds feasible trajectories which minimize the uncertainty of the volume estimate. We show results in simulation for th…
With the rapid advance of sophisticated control algorithms, the capabilities of drones to stabilise, fly and manoeuvre autonomously have dramatically improved, enabling us to pay greater attention to entire missions and the interaction of a drone with humans and with its environment during the course of such a mission. In this paper, we present an indoor office drone assistant that is tasked to run errands and carry out simple tasks at our laboratory, while given instructions from and interacting with humans in the space. To accomplish its mission, the system has to be able to understand verba…
Unmanned aerial vehicles (UAVs) are often used for navigating dangerous terrains, however they are difficult to pilot. Due to complex input-output mapping schemes, limited perception, the complex system dynamics and the need to maintain a safe operation distance, novice pilots experience difficulties in performing safe landings in obstacle filled environments. In this work we propose a shared autonomy approach that assists novice pilots to perform safe landings on one of several elevated platforms at a proficiency equal to or greater than experienced pilots. Our approach consists of two module…
Learning on the Fly: Replay-Based Continual Object Perception for Indoor Drones
National University of Science and Technology Politehnica Bucharest, Romania（罗马尼亚布加勒斯特多项技术大学）；Institute of Mathematics ``Simion Stoilow'' of the Romanian Academy, Romania（罗马尼亚科学院模拟昂斯数学研究所）；NORCE Norwegian Research Center, Norway（挪威NORCE研究中心）
This work focuses on drones or UAVs (Unmanned Aerial Vehicles) for use in industry in general. These vehicles have a large number of uses and potential in the industry, as a tool for civil engineering, medicine, mining, among others. However, this vehicle is limited for use indoors due to the need for GPS and it does not work indoors. In this way, this work presents a UAV that works without GPS, thus being able to be used in closed spaces for example and have good precision. The work is based on an approach that uses computer vision and GPS.
Pose Graph Optimization for a MAV Indoor Localization Fusing 5GNR TOA with an IMU
Interdisciplinary Center for Security Reliability and Trust (SnT)（安全可靠性与信任跨学科中心（SnT））；University of Luxembourg（卢森堡大学）；Faculty of Science, Technology, and Medicine (FSTM)（科学、技术与医学学院（FSTM））
本研究针对室内无人机定位问题，提出融合5G NR TOA与IMU数据的位姿图优化方法，通过仿真生成12组测试序列验证，在充足基站与高带宽配置下轨迹平均误差小于15厘米，且算法可实现机载实时跟踪。
Vision-based indoor localization of nano drones in controlled environment with its applications
Electrical Engineering Department, Indian Institute of Technology Bombay（印度理工学院班加罗尔分校电子工程系）；Embedded Real-Time Systems / e-Yantra Lab, Indian Institute of Technology Bombay（印度理工学院班加罗尔分校嵌入式实时系统/ e-Yantra实验室）；Computer Science and Engineering Department, Indian Institute of Technology Bombay（印度理工学院班加罗尔分校计算机科学与工程系）
Optimum Semiconductor Technologies Inc.（Optimum Semiconductor Technologies 公司）；University of Science and Technology Beijing（北京科技大学）
Unmanned aerial systems capable of beyond visual line of sight operation can be organized into a top-down hierarchy of layers including flight supervision, command and control, simulation of systems, operating systems, and physical hardware. Flight supervision includes unmanned air traffic management, flight planning, authorization, and remote identification. Command and control ensure drones can be piloted safely. Simulation of systems concerns how drones may react to different environments and how changing conditions and information provide input to a piloting system. Electronic hardware con…
DRAL: Deep Reinforcement Adaptive Learning for Multi-UAVs Navigation in Unknown Indoor Environment
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of California, Irvine（加利福尼亚大学欧文分校）；George Washington University（乔治·华盛顿大学）；Beijing Sineva Robot Technology Co., Ltd（北京思知机器人科技有限公司）；University of Southern California（南加利福尼亚大学）；Santa Clara University（圣克拉拉大学）；Mangosuthu University of Technology（曼戈苏图科技大学）
From Bench to Flight: Translating Drone Impact Tests into Operational Safety Limits
Lab INIT Robots, Department of Mechanical Engineering, École de technologie supérieure, Montreal, QC, Canada（Lab INIT Robots，机械工程系，École de technologie supérieure，蒙特利尔，QC，加拿大）
Zhejiang University（浙江大学）；Huzhou Institute of Zhejiang University（浙江大学湖州研究院）；DJI（大疆）
With the autonomy of aerial robots advances in recent years, autonomous drone racing has drawn increasing attention. In a professional pilot competition, a skilled operator always controls the drone to agilely avoid obstacles in aggressive attitudes, for reaching the destination as fast as possible. Autonomous flight like elite pilots requires planning in SE(3), whose non-triviality and complexity hindering a convincing solution in our community by now. To bridge this gap, this paper proposes an open-source baseline, which includes a high-performance SE(3) planner and a challenging simulation …
Robotics is the next frontier in the progress of Artificial Intelligence (AI), as the real world in which robots operate represents an enormous, complex, continuous state space with inherent real-time requirements. One extreme challenge in robotics is currently formed by autonomous drone racing. Human drone racers can fly through complex tracks at speeds of up to 190 km/h. Achieving similar speeds with autonomous drones signifies tackling fundamental problems in AI under extreme restrictions in terms of resources. In this article, we present the winning solution of the first AI Robotic Racing …
Ecole de technologie supérieure（高等技术学院）；Concordia University（康考迪亚大学）
The drone industry is diversifying and the number of pilots increases rapidly. In this context, flight schools need adapted tools to train pilots, most importantly with regard to their own awareness of their physiological and cognitive limits. In civil and military aviation, pilots can train themselves on realistic simulators to tune their reaction and reflexes, but also to gather data on their piloting behavior and physiological states. It helps them to improve their performances. Opposed to cockpit scenarios, drone teleoperation is conducted outdoor in the field, thus with only limited poten…
FleetScape: A Mixed Reality Sandtable for Spatial Supervision and Control of Scalable Drone Fleets
University of Toulouse, Fédération ENAC ISAE-SUPAERO ONERA（图卢兹大学、法国航空航天学校联合会、高等航空航天学院、法国航空航天研究院）；CNRS, IPAL（法国国家科学研究中心、IPAL机构）；National University of Singapore（新加坡国立大学）；National University of Singapore, Augmented Human Lab（新加坡国立大学增强人类实验室）；National University of Singapore, School of Computing（新加坡国立大学计算机学院）
We present NAVREN-RL, an approach to NAVigate an unmanned aerial vehicle in an indoor Real ENvironment via end-to-end reinforcement learning RL. A suitable reward function is designed keeping in mind the cost and weight constraints for micro drone with minimum number of sensing modalities. Collection of small number of expert data and knowledge based data aggregation is integrated into the RL process to aid convergence. Experimentation is carried out on a Parrot AR drone in different indoor arenas and the results are compared with other baseline technologies. We demonstrate how the drone succe…
Cargo drones are designed to carry payloads with predefined shape, size, and/or mass. This lack of flexibility requires a fleet of diverse drones tailored to specific cargo dimensions. Here we propose a new reconfigurable drone based on a modular design that adapts to different cargo shapes, sizes, and mass. We also propose a method for the automatic generation of drone configurations and suitable parameters for the flight controller. The parcel becomes the drone's body to which several individual propulsion modules are attached. We demonstrate the use of the reconfigurable hardware and the ac…
Robert Bosch Center for Cyberphysical Systems, Indian Institute of Science（印度科学理工学院罗伯特·博世网络物理系统中心）；Indian Institute of Technology Bombay（印度理工学院孟买分校）；Indian Institute of Technology Madras（印度理工学院马德拉斯分校）
Current trends in autonomous vehicles and their applications indicates an increasing need in positioning at low battery and compute cost. Lidars provide accurate localization at the cost of high compute and power consumption which could be detrimental for drones. Modern requirements for autonomous drones such as No-Permit-No-Takeoff (NPNT) and applications restricting drones to a corridor require the infrastructure to constantly determine the location of the drone. Ultra Wide Band Radios (UWB) fulfill such requirements and offer high precision localization and fast position update rates at a f…
Current drone delivery solutions mainly focus on single package delivery using one drone. However, the recent developments in drone technology enable a drone to deliver multiple packages in a single trip. We use the nearest destination first strategy for the faster delivery of packages in a skyway network. This demonstration is a proof-of-concept prototype for the multi-package delivery in urban airspace following a skyway network. We deploy and test this multi-package drone delivery in an indoor testbed environment using a 3D model of Sydney CBD. Demo: https://youtu.be/YTwsIfUvWPc
University of Zurich（苏黎世大学）；ETH Zurich（苏黎世联邦理工学院）；Intel（英特尔）
Performing acrobatic maneuvers with quadrotors is extremely challenging. Acrobatic flight requires high thrust and extreme angular accelerations that push the platform to its physical limits. Professional drone pilots often measure their level of mastery by flying such maneuvers in competitions. In this paper, we propose to learn a sensorimotor policy that enables an autonomous quadrotor to fly extreme acrobatic maneuvers with only onboard sensing and computation. We train the policy entirely in simulation by leveraging demonstrations from an optimal controller that has access to privileged in…
Zhejiang University（浙江大学）；University of Pennsylvania（宾夕法尼亚大学）
Current motion capture (MoCap) systems generally require markers and multiple calibrated cameras, which can be used only in constrained environments. In this work we introduce a drone-based system for 3D human MoCap. The system only needs an autonomously flying drone with an on-board RGB camera and is usable in various indoor and outdoor environments. A reconstruction algorithm is developed to recover full-body motion from the video recorded by the drone. We argue that, besides the capability of tracking a moving subject, a flying drone also provides fast varying viewpoints, which is beneficia…
Humans race drones faster than neural networks trained for end-to-end autonomous flight. This may be related to the ability of human pilots to select task-relevant visual information effectively. This work investigates whether neural networks capable of imitating human eye gaze behavior and attention can improve neural network performance for the challenging task of vision-based autonomous drone racing. We hypothesize that gaze-based attention prediction can be an efficient mechanism for visual information selection and decision making in a simulator-based drone racing task. We test this hypot…
Quadrotor drones equipped with high quality cameras have rapidely raised as novel, cheap and stable devices for filmmakers. While professional drone pilots can create aesthetically pleasing videos in short time, the smooth -- and cinematographic -- control of a camera drone remains challenging for most users, despite recent tools that either automate part of the process or enable the manual design of waypoints to create drone trajectories. This paper proposes to move a step further towards more accessible cinematographic drones by designing techniques to automatically or interactively plan qua…
Current drone delivery systems mostly focus on point-to-point package delivery. We present a multi-stop drone service system to deliver packages anywhere anytime within a specified geographic area. We define a skyway network which takes into account flying regulations, including restricted areas and no-fly zones. The skyway nodes typically represent building rooftops which may act as both recharging stations and delivery destinations. A heuristic-based A* algorithm is used to compute an optimal path from source to destination taking into account a number of constraints, including delivery time…
National Institute of Information and Communications Technology（国立信息通信技术研究所）
To meet the immediate needs of environmental monitoring or hazardous event detection, we consider the automatic deployment of a group of low-cost or disposable sensors by a drone. Introducing sensors by drones to an environment instead of humans has advantages in terms of worker safety and time requirements. In this study, we define "sensor scattering (SS)" as the problem of maximizing the information-theoretic gain from sensors scattered on the ground by a drone. SS is challenging due to its combinatorial explosion nature, because the number of possible combination of sensor positions increas…
Delft University of Technology（代尔夫特理工大学）；University of Liverpool（利物浦大学）；Radboud University of Nijmegen（奈梅亨拉德堡德大学）
Miniature Micro Aerial Vehicles (MAV) are very suitable for flying in indoor environments, but autonomous navigation is challenging due to their strict hardware limitations. This paper presents a highly efficient computer vision algorithm called Edge-FS for the determination of velocity and depth. It runs at 20 Hz on a 4 g stereo camera with an embedded STM32F4 microprocessor (168 MHz, 192 kB) and uses feature histograms to calculate optical flow and stereo disparity. The stereo-based distance estimates are used to scale the optical flow in order to retrieve the drone's velocity. The velocity …
Drone racing is becoming a popular e-sport all over the world, and beating the best human drone race pilots has quickly become a new major challenge for artificial intelligence and robotics. In this paper, we propose a strategy for autonomous drone racing which is computationally more efficient than navigation methods like visual inertial odometry and simultaneous localization and mapping. This fast light-weight vision-based navigation algorithm estimates the position of the drone by fusing race gate detections with model dynamics predictions. Theoretical analysis and simulation results show t…
University of Oxford（牛津大学）；University of Edinburgh（爱丁堡大学）
Accurate motion capture of aerial robots in 3-D is a key enabler for autonomous operation in indoor environments such as warehouses or factories, as well as driving forward research in these areas. The most commonly used solutions at present are optical motion capture (e.g. VICON) and Ultrawideband (UWB), but these are costly and cumbersome to deploy, due to their requirement of multiple cameras/sensors spaced around the tracking area. They also require the drone to be modified to carry an active or passive marker. In this work, we present an inexpensive system that can be rapidly installed, b…
Over 600,000 bridges in the U.S. must be inspected every two years to identify flaws, defects, or potential problems that may need follow-up maintenance. Bridge inspection has adopted unmanned aerial vehicles (or drones) for improving safety, efficiency, and cost-effectiveness. Although drones can operate in an autonomous mode, keeping inspectors in the loop is critical for complex tasks in bridge inspection. Therefore, inspectors need to develop the skill and confidence to operate drones in their jobs. This paper presents the design and development of a virtual reality-based training and asse…
University of Zurich（苏黎世大学）；ETH Zurich（苏黎世联邦理工学院）；Intel Labs（英特尔实验室）
Autonomous agile flight brings up fundamental challenges in robotics, such as coping with unreliable state estimation, reacting optimally to dynamically changing environments, and coupling perception and action in real time under severe resource constraints. In this paper, we consider these challenges in the context of autonomous, vision-based drone racing in dynamic environments. Our approach combines a convolutional neural network (CNN) with a state-of-the-art path-planning and control system. The CNN directly maps raw images into a robust representation in the form of a waypoint and desired…
This paper presents a new framework to use images as the inputs for the controller to have autonomous flight, considering the noisy indoor environment and uncertainties. A new Proportional-Integral-Derivative-Accelerated (PIDA) control with a derivative filter is proposed to improves drone/quadcopter flight stability within a noisy environment and enables autonomous flight using object and depth detection techniques. The mathematical model is derived from an accurate model with a high level of fidelity by addressing the problems of non-linearity, uncertainties, and coupling. The proposed PIDA …
