This paper introduces a safe swarm of drones capable of performing landings in crowded environments robustly by relying on Reinforcement Learning techniques combined with Safe Learning. The developed system allows us to teach the swarm of drones with different dynamics to land on moving landing pads in an environment while avoiding collisions with obstacles and between agents.
The safe barrier net algorithm was developed and evaluated using a swarm of Crazyflie 2.1 micro quadrotors, which were tested indoors with the Vicon motion capture system to ensure precise localization and control.
Experimental results show that our system achieves landing accuracy of 2.25 cm with a mean time of 17 s and collision-free landings, underscoring its effectiveness and robustness in real-world scenarios. This work offers a promising foundation for applications in environments where safety and precision are paramount.
Landing is the most challenging and risky aspect of multirotor drone flight, and only simple landing methods exist for autonomous drones. We explore methods for autonomous drone landing in two scenarios. In the first scenario, we examine methods for landing on known landing pads using fiducial markers and a gimbal-mounted monocular camera. This method has potential in drone applications where a drone must land more accurately than GPS can provide (e.g.~package delivery in an urban canyon). We expand on previous methods by actuating the drone's camera to track the marker over time, and we addre…
Currently, drone research and development has received significant attention worldwide. Particularly, delivery services employ drones as it is a viable method to improve delivery efficiency by using a several unmanned drones. Research has been conducted to realize complete automation of drone control for such services. However, regarding the takeoff and landing port of the drones, conventional methods have focused on the landing operation of a single drone, and the continuous landing of multiple drones has not been realized. To address this issue, we propose a completely novel port system, "EA…
The paper focuses on a heterogeneous swarm of drones to achieve a dynamic landing of formation on a moving robot. This challenging task was not yet achieved by scientists. The key technology is that instead of facilitating each agent of the swarm of drones with computer vision that considerably increases the payload and shortens the flight time, we propose to install only one camera on the leader drone. The follower drones receive the commands from the leader UAV and maintain a collision-free trajectory with the artificial potential field. The experimental results revealed a high accuracy of t…
mmE-Loc: Facilitating Accurate Drone Landing with Ultra-High-Frequency Localization
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Carnegie Mellon University（卡内基梅隆大学）；Meituan Academy of Robotics Shenzhen and Meituan Inc.（美团机器人研究院（深圳）及美团公司）；School of Software and BNRist, Tsinghua University（清华大学软件学院）；Shenzhen Smart City Communication Co., Ltd.（深圳智慧城市通信有限公司）；Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；International Digital Economy Academy（国际数字经济 academy）
Enabling High-Frequency Cross-Modality Visual Positioning Service for Accurate Drone Landing
Shenzhen International Graduate School, Tsinghua University, China（清华大学深圳国际研究生院）；Carnegie Mellon University, USA（卡内基梅隆大学）；Meituan Academy of Robotics Shenzhen and Meituan Inc., China（美团机器人深圳研究院和美团公司）；Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China, Shenzhen, China（电子科技大学深圳研究院）；School of Software and BNRist, Tsinghua University, Beijing 100084, China（软件学院和BNRist，清华大学，北京100084，中国）
Unmanned aerial vehicles (UAVs) are often used for navigating dangerous terrains, however they are difficult to pilot. Due to complex input-output mapping schemes, limited perception, the complex system dynamics and the need to maintain a safe operation distance, novice pilots experience difficulties in performing safe landings in obstacle filled environments. In this work we propose a shared autonomy approach that assists novice pilots to perform safe landings on one of several elevated platforms at a proficiency equal to or greater than experienced pilots. Our approach consists of two module…
Precision landing is a remaining challenge in autonomous drone flight. Fiducial markers provide a computationally cheap way for a drone to locate a landing pad and autonomously execute precision landings. However, most work in this field depends on either rigidly-mounted or downward-facing cameras which restrict the drone's ability to detect the marker. We present a method of autonomous landing that uses a gimbal-mounted camera to quickly search for the landing pad by simply spinning in place while tilting the camera up and down, and to continually aim the camera at the landing pad during appr…
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）
Heterogeneous teams of mobile robots and UAVs are offering a substantial benefit in an autonomous exploration of the environment. Nevertheless, although joint exploration scenarios for such systems are widely discussed, they are still suffering from low adaptability to changes in external conditions and faults of swarm agents during the UAV docking. We propose a novel vision-based drone swarm docking system for robust landing on a moving platform when one of the agents lost its position signal. The proposed SwarmHawk system relies on vision-based detection for the mobile platform tracking and …
Mila - Quebec Institute of Artificial Intelligence, Université de Montréal（米拉-魁北克人工智能研究所，蒙特利尔大学）；Alternova Tech SAS（Alternova科技公司）；Universidad Autónoma de Occidente（西部自治大学）
Autonomous landing is a capability that is essential to achieve the full potential of multi-rotor drones in many social and industrial applications. The implementation and testing of this capability on physical platforms is risky and resource-intensive; hence, in order to ensure both a sound design process and a safe deployment, simulations are required before implementing a physical prototype. This paper presents the development of a monocular visual system, using a software-in-the-loop methodology, that autonomously and efficiently lands a quadcopter drone on a predefined landing pad, thus r…
Ultra-High-Frequency Harmony: mmWave Radar and Event Camera Orchestrate Accurate Drone Landing
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Carnegie Mellon University（卡内基梅隆大学）；Meituan Academy of Robotics Shenzhen（美团深圳机器人研究院）；School of Software, Tsinghua University（清华大学软件学院）；Pengcheng Laboratory（鹏城实验室）；RISC-V International Open Source Laboratory（RISC-V国际开源实验室）
LLM-Land: Large Language Models for Context-Aware Drone Landing
Department of Electrical and Computer Engineering（电气与计算机工程系）；Drexel University（德雷塞尔大学）；Department of Electrical and Systems Engineering（电气与系统工程系）；University of Pennsylvania（宾夕法尼亚大学）
University of ElectroCommunications, Tokyo（东京电机大学）
For the human operator, it is often easier and faster to catch a small size quadrotor right in the midair instead of landing it on a surface. However, interaction strategies for such cases have not yet been considered properly, especially when more than one drone has to be landed at the same time. In this paper, we propose a novel interaction strategy to land multiple robots on the human hands using vibrotactile feedback. We developed a wearable tactile display that is activated by the intensity of light emitted from an LED ring on the bottom of the quadcopter. We conducted experiments, where …
We propose a novel system SwarmCloak for landing of a fleet of four flying robots on the human arms using light-sensitive landing pads with vibrotactile feedback. We developed two types of wearable tactile displays with vibromotors which are activated by the light emitted from the LED array at the bottom of quadcopters. In a user study, participants were asked to adjust the position of the arms to land up to two drones, having only visual feedback, only tactile feedback or visual-tactile feedback. The experiment revealed that when the number of drones increases, tactile feedback plays a more i…
California Institute of Technology（加州理工学院）；Northeastern University（东北大学）；University of California, Irvine（加州大学欧文分校）
Precise near-ground trajectory control is difficult for multi-rotor drones, due to the complex aerodynamic effects caused by interactions between multi-rotor airflow and the environment. Conventional control methods often fail to properly account for these complex effects and fall short in accomplishing smooth landing. In this paper, we present a novel deep-learning-based robust nonlinear controller (Neural Lander) that improves control performance of a quadrotor during landing. Our approach combines a nominal dynamics model with a Deep Neural Network (DNN) that learns high-order interactions.…
Technion Israel Institute of Technology（以色列理工学院）；Rafael Advanced Defense Systems Ltd.（拉斐尔先进防御系统有限公司）
This paper considers the problem of finding a landing spot for a drone in a dense urban environment. The conflicting requirement of fast exploration and high resolution is solved using a multi-resolution approach, by which visual information is collected by the drone at decreasing altitudes so that spatial resolution of the acquired images increases monotonically. A probability distribution is used to capture the uncertainty of the decision process for each terrain patch. The distributions are updated as information from different altitudes is collected. When the confidence level for one of th…
With the development of industry, drones are appearing in various field. In recent years, deep reinforcement learning has made impressive gains in games, and we are committed to applying deep reinforcement learning algorithms to the field of robotics, moving reinforcement learning algorithms from game scenarios to real-world application scenarios. We are inspired by the LunarLander of OpenAI Gym, we decided to make a bold attempt in the field of reinforcement learning to control drones. At present, there is still a lack of work applying reinforcement learning algorithms to robot control, the p…
University of Maryland Institute for Advanced Computer Studies（马里兰大学高级计算机研究所）；University of Maryland, College Park（马里兰大学帕克分校）；Delft University of Technology（代尔夫特理工大学）
The rapid rise of accessibility of unmanned aerial vehicles or drones pose a threat to general security and confidentiality. Most of the commercially available or custom-built drones are multi-rotors and are comprised of multiple propellers. Since these propellers rotate at a high-speed, they are generally the fastest moving parts of an image and cannot be directly "seen" by a classical camera without severe motion blur. We utilize a class of sensors that are particularly suitable for such scenarios called event cameras, which have a high temporal resolution, low-latency, and high dynamic rang…
Visual-based Safe Landing for UAVs in Populated Areas: Real-time Validation in Virtual Environments
Universidad Autónoma de Ciudad Juárez UACJ（华雷斯自治大学）；Center for Research in Mathematics CIMAT AC（数学研究中心CIMAT AC）；University of Texas at El Paso（德克萨斯大学埃尔帕索分校）
Landing is a challenging part of autonomous drone flight and a great research opportunity. This PhD proposes to improve on fiducial autonomous landing algorithms by making them more flexible. Further, it leverages its location, Iceland, to develop a method for landing on lava flows in cooperation with analog Mars exploration missions taking place in Iceland now - and potentially for future Mars landings.
Safe autonomous landing in urban cities is a necessity for the growing Unmanned Aircraft Systems (UAS) industry. In urgent situations, building rooftops, particularly flat rooftops, can provide local safe landing zones for small UAS. This paper investigates the real-time identification and selection of safe landing zones on rooftops based on LiDAR and camera sensor feedback. A visual high fidelity simulated city is constructed in the Unreal game engine, with particular attention paid to accurately generating rooftops and the common obstructions found thereon, e.g., ac units, water towers, air …
Drone swarms are teams of autonomous un-manned aerial vehicles that act as a collective entity. We are interested in humanizing drone swarms, equipping them with the ability to emotionally affect human users through their non-verbal motions. Inspired by recent findings in how observers are emotionally touched by watching dance moves, we investigate the questions of whether and how coordinated drone swarms' motions can achieve emotive impacts on general audience. Our preliminary study on Amazon Mechanical Turk led to a number of interesting findings, including both promising results and challen…
Center for Research in Mathematics CIMAT AC（数学研究中心（CIMAT AC））
Autonomous landing of Unmanned Aerial Vehicles (UAVs) in crowded scenarios is crucial for successful deployment of UAVs in populated areas, particularly in emergency landing situations where the highest priority is to avoid hurting people. In this work, a new visual-based algorithm for identifying Safe Landing Zones (SLZ) in crowded scenarios is proposed, considering a camera mounted on an UAV, where the people in the scene move with unknown dynamics. To do so, a density map is generated for each image frame using a Deep Neural Network, from where a binary occupancy map is obtained aiming to o…
As UAV technology improves, more uses have been found for these versatile autonomous vehicles, from surveillance to aerial photography, to package delivery, and each of these applications poses unique challenges. This paper implements a solution for one such challenge: To land on a moving target. This problem has been addressed before with varying degrees of success, however, most implementations focus on indoor applications. Outdoor poses greater challenges in the form of variables such as wind and lighting, and outdoor drones are heavier and more susceptible to inertial effects. Our approach…
KIOS Research and Innovation Center of Excellence（KIOS卓越研究与创新中心）；University of Cyprus（塞浦路斯大学）
While Unmanned Aerial Vehicles (UAVs) are increasingly deployed in several missions, their inability of reliable and consistent autonomous landing poses a major setback for deploying such systems truly autonomously. In this paper we present an autonomous UAV landing system for landing on a moving platform. In contrast to existing attempts, the proposed system relies only on the camera sensor, and has been designed as lightweight as possible. The proposed system can be deployed on a low power platform as part of the drone payload, whilst being indifferent to any external communication or any ot…
A Modular Pneumatic Soft Gripper Design for Aerial Grasping and Landing
The Hong Kong Polytechnic University（香港理工大学）；Hong Kong Center for Construction Robotics（香港建造机器人中心）；Research Centre of Unmanned Autonomous Systems（无人自主系统研究中心）；Department of Aeronautical and Aviation Engineering（航空及民航工程系）；Department of Mechanical Engineering（机械工程系）
Rigid grippers used in existing aerial manipulators require precise positioning to achieve successful grasps and transmit large contact forces that may destabilize the drone. This limits the speed during grasping and prevents "dynamic grasping", where the drone attempts to grasp an object while moving. On the other hand, biological systems (e.g., birds) rely on compliant and soft parts to dampen contact forces and compensate for grasping inaccuracy, enabling impressive feats. This paper presents the first prototype of a soft drone -- a quadrotor where traditional (i.e., rigid) landing gears ar…
Omnidirectional Dual-Arm Aerial Manipulator with Proprioceptive Contact Localization for Landing on Slanted Roofs
BioMorphic Intelligence Lab, Dept. Control & Operations, Faculty of Aerospace Engineering, TU Delft（德鲁特技术大学航空航天工程学院控制与运营系生物形态智能实验室）；School of Engineering Mathematics and Technology, and Bristol Robotics Laboratory, University of Bristol（布里斯托大学工程数学与技术学院及布里斯托机器人实验室）
Texas A&M University – Commerce（得克萨斯农工大学康莫斯分校）；Fordham University（福特汉姆大学）
Drones are conventionally controlled using joysticks, remote controllers, mobile applications, and embedded computers. A few significant issues with these approaches are that drone control is limited by the range of electromagnetic radiation and susceptible to interference noise. In this study we propose the use of hand gestures as a method to control drones. We investigate the use of computer vision methods to develop an intuitive way of agent-less communication between a drone and its operator. Computer vision-based methods rely on the ability of a drone's camera to capture surrounding image…
This paper describes an approach to building a cost-effective and research grade visual-inertial odometry aided vertical taking-off and landing (VTOL) platform. We utilize an off-the-shelf visual-inertial sensor, an onboard computer, and a quadrotor platform that are factory-calibrated and mass-produced, thereby sharing similar hardware and sensor specifications (e.g., mass, dimensions, intrinsic and extrinsic of camera-IMU systems, and signal-to-noise ratio). We then perform a system calibration and identification enabling the use of our visual-inertial odometry, multi-sensor fusion, and mode…
Soft Haptics Lab., School of Materials Science, Japan Advanced Institute of Science and Technology（日本先进科学技术学院材料科学学院软触觉实验室）；Tandon School of Engineering, New York University（纽约大学坦登工程学院）；Japan Science and Technology Agency, PRESTO（日本科学技术厅）
There is a growing need for vertical take-off and landing vehicles, including drones, which are safe to use and can adapt to collisions. The risks of damage by collision, to humans, obstacles in the environment, and drones themselves, are significant. This has prompted a search into nature for a highly resilient structure that can inform a design of propellers to reduce those risks and enhance safety. Inspired by the flexibility and resilience of dragonfly wings, we propose a novel design for a biomimetic drone propeller called Tombo propeller. Here, we report on the design and fabrication pro…
Crash Landing onto "you": Untethered Soft Aerial Robots for Safe Environmental Interaction, Sensing, and Perching
Empa (Swiss Federal Laboratories for Materials Science and Technology)（瑞士联邦材料科学与技术研究所（Empa））；Imperial College London（伦敦帝国学院）；RSS Pioneers 2024（RSS先锋2024）
Risk Assessment for Autonomous Landing in Urban Environments using Semantic Segmentation
Center for Research in Mathematics, CIMAT AC（墨西哥数学研究中心（CIMAT AC））；Centro de Investigación en Matemáticas, A.C.（墨西哥数学研究中心）；Centro de Investigaciones en Óptica, A.C.（墨西哥光学研究中心）；CONAHCYT（墨西哥国家科学技术委员会）
Vision-based indoor localization of nano drones in controlled environment with its applications
Electrical Engineering Department, Indian Institute of Technology Bombay（印度理工学院班加罗尔分校电子工程系）；Embedded Real-Time Systems / e-Yantra Lab, Indian Institute of Technology Bombay（印度理工学院班加罗尔分校嵌入式实时系统/ e-Yantra实验室）；Computer Science and Engineering Department, Indian Institute of Technology Bombay（印度理工学院班加罗尔分校计算机科学与工程系）
The flight characteristics of bats remarkably have been overlooked in aerial drone designs. Unlike other animals, bats leverage the manipulation of inertial dynamics to exhibit aerial flip turns when they perch. Inspired by this unique maneuver, this work develops and uses a tiny robot called \textit{Harpoon} to demonstrate that the preparation for upside-down landing is possible through: 1) reorientation towards the landing surface through zero-angular-momentum turns and 2) reaching to the surface through shooting a detachable landing gear. The closed-loop manipulations of inertial dynamics t…
Sun Yat-sen University（中山大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；Hong Kong University of Science and Technology（香港科学大学）；City University of Hong Kong（香港城市大学）
University of Maryland, College Park（马里兰大学帕克分校）；Institute for Systems Research（系统研究所）
We explore an end-to-end (including access and backhaul links) UAV-assisted wireless communication system, considering both uplink and downlink traffics, with the goal of supporting demand of the Ground Users (GUs) using the minimum number of UAVs. Moreover, in order to extend the operational (flight) time of UAVs, we exploit an energy-aware routing scheme. Our intention is to design and analyze the access and backhaul connectivity of a drone-assisted communication network for remote and crowded areas and disaster relief, while minimizing the resources required i.e., the number of UAVs.
An Experimental Study of Trojan Vulnerabilities in UAV Autonomous Landing
Department of Electrical and Computer Engineering at North Carolina A&T State University（北卡罗来纳A&T州立大学电气与计算机工程系）；University of Alabama（阿拉巴马大学）；Department of Electrical and Computer Engineering, Windsor University（温莎大学电气与计算机工程系）
