We propose a novel filter for segmenting the regions of interest from LiDAR 3D point cloud for multirotor aerial vehicles. It is specially targeted for real-time applications and works on sparse LiDAR point clouds without preliminary mapping. We use this filter as a crucial component of fast obstacle avoidance system for agriculture drone operating at low altitude. As the first step, each point cloud is transformed into a depth image and then identify places near to the vehicle (local maxima) by locating areas with high pixel densities. Afterwards, we merge the original depth image with identified locations after maximizing intensities of pixels in which local maxima were obtained. Next step is to calculate the range angle image that represents angles between two consecutive laser beams based on the improved depth image. Once the corresponding range angle image is constructed, smoothing is applied to reduce the noise. Finally, we find out connected components within the improved depth image while incorporating smoothed range angle image. This allows separating the regions of interest. The filter has been tested on various simulated environments as well as an actual drone and provides real-time performance. We make our source code, dataset available at https://github.com/GPrathap/hagen.git and real world experiment result can be found on the following link: https://www.youtube.com/watch?v=iHd_ZkhKPjc available online.
Ford AV LLC（福特自动驾驶有限责任公司）；Ford Motor Company（福特汽车公司）
This paper presents a localization technique using aerial imagery maps and LIDAR based ground reflectivity for autonomous vehicles in urban environments. Traditional localization techniques using LIDAR reflectivity rely on high definition reflectivity maps generated from a mapping vehicle. The cost and effort required to maintain such prior maps are generally very high because it requires a fleet of expensive mapping vehicles. In this work we propose a localization technique where the vehicle localizes using aerial/satellite imagery, eradicating the need to develop and maintain complex high-de…
This paper considers a class of rotary-wing aerial robots with unaligned propellers. By studying the dynamics of these vehicles, we show that the position and heading angle remain flat outputs of the system (similar to conventional quadrotors). The implication is that they can be commanded to follow desired trajectory setpoints in 3D space. We propose a numerical strategy based on the collocation method to facilitate the trajectory generation. This enables convenient computation of the nominal robot's attitude and control inputs. The proposed methods are numerically verified for three multirot…
Universidad de Sevilla（塞维利亚大学）；University of Zurich（苏黎世大学）；ETH Zurich（苏黎世联邦理工学院）
Multirotor aerial robots are becoming widely used for the inspection of powerlines. To enable continuous, robust inspection without human intervention, the robots must be able to perch on the powerlines to recharge their batteries. Highly versatile perching capabilities are necessary to adapt to the variety of configurations and constraints that are present in real powerline systems. This paper presents a novel perching trajectory generation framework that computes perception-aware, collision-free, and dynamically-feasible maneuvers to guide the robot to the desired final state. Trajectory gen…
An Experimental Study of Downwash Effects on a Continuum Manipulator Integrated with a Multirotor UAV
Queensland University of Technology (QUT)（昆士兰科技大学）；QUT Centre for Robotics（昆士兰科技大学机器人中心）；Research Engineering Facility (REF)（研究工程设施）；School of Electrical Engineering and Robotics（电气工程与机器人学院）
We present SMORS, the first Soft fully actuated MultirOtoR System for multimodal locomotion. Unlike conventional hexarotors, SMORS is equipped with three rigid and three continuously soft arms, with each arm hosting a propeller. We create a bridge between the fields of soft and aerial robotics by mechanically coupling the actuation of a fully actuated flying platform with the actuation of a soft robotic manipulator. Each rotor is slightly tilted, allowing for full actuation of the platform. The soft components combined with the platform's full actuation allow for a robust interaction, in the f…
Co-Registration of aerial imagery and Light Detection and Ranging (LiDAR) data is quilt challenging because the different imaging mechanism causes significant geometric and radiometric distortions between such data. To tackle the problem, this paper proposes an automatic registration method based on structural features and three-dimension (3D) phase correlation. In the proposed method, the LiDAR point cloud data is first transformed into the intensity map, which is used as the reference image. Then, we employ the Fast operator to extract uniformly distributed interest points in the aerial imag…
Recently, needs for unmanned aerial vehicles (UAVs) that are attachable to the wall have been highlighted. As one of the ways to address the need, researches on various tilting multirotors that can increase maneuverability has been employed. Unfortunately, existing studies on the tilting multirotors require considerable amounts of prior information on the complex dynamic model. Meanwhile, reinforcement learning on quadrotors has been studied to mitigate this issue. Yet, these are only been applied to standard quadrotors, whose systems are less complex than those of tilting multirotors. In this…
This paper presents a new method for integrated time-optimal routing and trajectory optimization of multirotor unmanned aerial vehicles (UAVs). Our approach extends the well-known Traveling Salesman Problem by accounting for the limited maneuverability of the UAVs due to their kinematic properties. To this end, we allow each waypoint to be traversed with a discretized velocity as well as a discretized flight direction and compute time-optimal trajectories to determine the travel time costs for each edge. We refer to this novel optimization problem as the Trajectory-based Traveling Salesman Pro…
Localizing radio frequency (RF) sources with an unmanned aerial vehicle (UAV) has many important applications. As a result, UAV-based localization has been the focus of much research. However, previous approaches rely heavily on custom electronics and specialized knowledge, are not robust and require extensive calibration, or are inefficient with measurements and waste energy on a battery-constrained platform. In this work, we present a system based on a multirotor UAV that addresses these shortcomings. Our system measures signal strength received by two antennas to update a probability distri…
As platform size is reduced, the flight of aerial robots becomes increasingly energetically expensive. Limitations on payload and endurance of these small robots have prompted researchers to explore the use of bimodal aerial-surface locomotion as a strategy to prolong operation time while retaining a high vantage point. In this work, we propose the use of ``ceiling effects'' as a power conserving strategy for small rotorcraft to perch on an overhang. In the vicinity of a ceiling, spinning propellers generate markedly higher thrust. To understand the observed aerodynamic phenomena}, momentum th…
Australian Center for Field Robotics, the University of Sydney（悉尼大学澳大利亚野外机器人中心）
Centimeter level globally accurate and consistent maps for autonomous vehicles navigation has long been achieved by on board real-time kinematic(RTK)-GPS in open areas. However when dealing with urban environments, GPS will experience multipath and blockage in urban canyon, under bridges, inside tunnels and in underground environments. In this paper we present strategies to efficiently register local maps in geographical coordinate systems through the tactical integration of GPS and information extracted from precisely geo-referenced high resolution aerial orthogonal imagery. Dense lidar point…
Adaptive RISE Control for Dual-Arm Unmanned Aerial Manipulator Systems with Deep Neural Networks
Nankai University（南开大学）；College of Artificial Intelligence（人工智能学院）；Institute of Robotics and Automatic Information System（机器人与自动信息系统研究所）；Engineering Research Center of Trusted Behavior Intelligence, Ministry of Education（教育部可信行为智能工程研究中心）
Fusion of Visual-Inertial Odometry with LiDAR Relative Localization for Cooperative Guidance of a Micro-Scale Aerial Vehicle
Multi-robot Systems Group, Department of Cybernetics, Faculty of Electrical Engineering, Czech Technical University in Prague, Technick\'a 2, Prague, 166 27, Czech Republic
This paper investigates the use of LiDAR SLAM as a pose feedback for autonomous flight. Cartographer, LOAM and HDL graph SLAM are first introduced on a conceptual level and later tested for this role. They are first compared offline on a series of datasets to see if they are capable of producing high-quality pose estimates in agile and long-range flight scenarios. The second stage of testing consists of integrating the SLAM algorithms into a cascade PID UAV control system and comparing the control system performance on step excitation signals and helical trajectories. The comparison is based o…
Design, Modeling and Control of a Top-loading Fully-Actuated Cargo Transportation Multirotor
Seoul National University of Science and Technology (SEOULTECH)（首尔科学技术大学（서울과학기술대학교））；Autel Robotics（道通智能）；Seoul National University (SNU)（首尔大学）；Automation and Systems Research Institute (ASRI)（自动化与系统研究所）
Physics-Informed Neural Network for Multirotor Slung Load Systems Modeling
Institute for Systems and Robotics (ISR)（系统与机器人研究所）；Laboratory of Robotics and Engineering Systems (LARSyS)（机器人与工程系统实验室）；Instituto Superior Técnico（高等技术学院）；University of Lisbon（里斯本大学）；CTS/Uninova（新大学中心技术与系统研究中心）；NOVA School of Science and Technology (FCT-NOVA)（NOVA科学与技术学院）
In many unmanned aerial vehicle (UAV) applications for surveillance and data collection, it is not possible to reach all requested locations due to the given maximum flight time. Hence, the requested locations must be prioritized and the problem of selecting the most important locations is modeled as an Orienteering Problem (OP). To fully exploit the kinematic properties of the UAV in such scenarios, we combine the OP with the generation of time-optimal trajectories with bounds on velocity and acceleration. We define the resulting problem as the Kinematic Orienteering Problem (KOP) and propose…
In recent years, autonomous robots have become ubiquitous in research and daily life. Among many factors, public datasets play an important role in the progress of this field, as they waive the tall order of initial investment in hardware and manpower. However, for research on autonomous aerial systems, there appears to be a relative lack of public datasets on par with those used for autonomous driving and ground robots. Thus, to fill in this gap, we conduct a data collection exercise on an aerial platform equipped with an extensive and unique set of sensors: two 3D lidars, two hardware-synchr…
The demand for accurate and fast trajectory tracking for multirotor Unmanned Aerial Vehicles (UAVs) have grown recently due to advances in UAV avionics technology and application domains. In many applications, the multirotor UAV is required to accurately perform aggressive maneuvers in challenging scenarios like the presence of external wind disturbances or in-flight payload changes. In this paper, we propose a systematic controller tuning approach based on identification results obtained by a recently developed Deep Neural Networks with the Modified Relay Feedback Test (DNN-MRFT) algorithm. W…
Cross-Session 3D LiDAR and Camera Fusion for Robust Localization of Unmanned Aerial Vehicles in GPS-Denied Environments
School of Electrical and Data Engineering, University of Technology Sydney（悉尼大学电气与数据工程学院）；Institute of Aerospace Engineering and Technology, Duy Tan University（杜益大学航空航天工程与技术学院）；College of Engineering and Computer Science, VinUniversity（Vin大学工程与计算机科学学院）；Smart Green Transformation Center (GREEN-X), VinUniversity（Vin大学智能绿色转型中心 (GREEN-X)）；Department of Artificial Intelligence and Robotics, Sejong University（成均馆大学人工智能与机器人系）
Visibility-Aware Cooperative Tracking with Decentralized LiDAR-Based Aerial Swarms
Department of Mechanical Engineering, The University of Hong Kong（香港大学机械工程系）；School of Intelligent System Engineering, Sun Yat-sen University（中山大学智能系统工程学院）
Energy sources such as batteries do not decrease in mass after consumption, unlike combustion-based fuels. We present the concept of staging energy sources, i.e. consuming energy in stages and ejecting used stages, to progressively reduce the mass of aerial vehicles in-flight which reduces power consumption, and consequently increases flight time. A flight time vs. energy storage mass analysis is presented to show the endurance benefit of staging to multirotors. We consider two specific problems in discrete staging -- optimal order of staging given a certain number of energy sources, and optim…
Center for Technologies in Robotics and Mechatronics Components, Innopolis University（英诺波利斯大学机器人与机电一体化组件技术中心）
We explore path planning followed by kinodynamic smoothing while ensuring the vehicle dynamics feasibility for MAVs. We have chosen a geometrically based motion planning technique \textquotedblleft RRT*\textquotedblright\; for this purpose. In the proposed technique, we modified original RRT* introducing an adaptive search space and a steering function which help to increase the consistency of the planner. Moreover, we propose multiple RRT* which generates a set of desired paths, provided that the optimal path is selected among them. Then, apply kinodynamic smoothing, which will result in dyna…
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；The University of Tokyo（东京大学）；Line Corporation（Line公司）
Estimating the pose of an unmanned aerial vehicle (UAV) or drone is a challenging task. It is useful for many applications such as navigation, surveillance, tracking objects on the ground, and 3D reconstruction. In this work, we present a LiDAR-camera-based relative pose estimation method between a drone and a ground vehicle, using a LiDAR sensor and a fisheye camera on the vehicle's roof and another fisheye camera mounted under the drone. The LiDAR sensor directly observes the drone and measures its position, and the two cameras estimate the relative orientation using indirect observation of …
Integration of a Variable Stiffness Link for Long-Reach Aerial Manipulation
Department of Electrical and Photonics Engineering Control, Robotics and Embodied AI, Technical University of Denmark（技术大学丹麦分校电气与光子工程系）；GRVC Robotics Laboratory, Universidad de Sevilla（塞维利亚大学GRVC机器人实验室）
This letter suggests an integrated approach for a drone (or multirotor) to perform an autonomous videography task in a 3-D obstacle environment by following a moving object. The proposed system includes 1) a target motion prediction module which can be applied to dense environments and 2) a hierarchical chasing planner based on a proposed metric for visibility. In the prediction module, we minimize observation error given that the target object itself does not collide with obstacles. The estimated future trajectory of target is obtained by covariant optimization. The other module, chasing plan…
Unmanned aerial vehicles (UAVs) equipped with multiple complementary sensors have tremendous potential for fast autonomous or remote-controlled semantic scene analysis, e.g., for disaster examination. In this work, we propose a UAV system for real-time semantic inference and fusion of multiple sensor modalities. Semantic segmentation of LiDAR scans and RGB images, as well as object detection on RGB and thermal images, run online onboard the UAV computer using lightweight CNN architectures and embedded inference accelerators. We follow a late fusion approach where semantic information from mult…
FlyAware: Inertia-Aware Aerial Manipulation via Vision-Based Estimation and Post-Grasp Adaptation
School of Intelligent Systems Engineering, Sun Yat-sen University（中山大学智能系统工程学院）；Visual Intelligence Lab, Cheng Kar-Shun Robotics Institute, The Hong Kong University of Science and Technology（香港科技大学成卡顺机器人研究所视觉智能实验室）；Differential Robotics Technology Company, Ltd.（差分机器人技术有限公司）
Energy management is a critical aspect of risk assessment for Uncrewed Aerial Vehicle (UAV) flights, as a depleted battery during a flight brings almost guaranteed vehicle damage and a high risk of human injuries or property damage. Predicting the amount of energy a flight will consume is challenging as routing, weather, obstacles, and other factors affect the overall consumption. We develop a deep energy model for a UAV that uses Temporal Convolutional Networks to capture the time varying features while incorporating static contextual information. Our energy model is trained on a real world d…
