Altitude-Adaptive Vision-Only Geo-Localization for UAVs in GPS-Denied Environments
Department of Precision Instrument, Tsinghua University（清华大学精密仪器系）；School of Aerospace Engineering, Beijing Institute of Technology（北京理工大学航天工程学院）；School of Instrumentation Science and Opto-electronics Engineering, Beijing Information Science and Technology University（北京信息科技大学仪器科学与光电工程学院）
Matching downward-looking unmanned aerial vehicle (UAV) images to georeferenced satellite or aerial map tiles supports local earth observation, map-based interpretation, and coarse geo-initialization when direct positioning signals are degraded or unavailable. A major difficulty in this cross-platform matching task is the scale mismatch caused by large variations in UAV altitude. To address this problem, we propose an altitude-adaptive aerial visual place recognition framework. The method first estimates relative altitude from a single downward-looking image by transforming the input into the frequency domain and formulating altitude estimation as a regression-as-classification problem. The estimated altitude is then used to crop the query image to a canonical scale, after which a classification-then-retrieval visual place recognition module performs coarse map localization. To improve retrieval robustness under varying image quality, we further introduce a quality-adaptive margin classifier and refine the final location by weighted coordinate estimation over the top retrieved candidates. Experiments on two synthetic datasets and two real-flight datasets show that the relative altitude estimation module improves downstream retrieval performance under substantial altitude changes. With our visual place recognition module, altitude adaptation improves average R@1 and R@5 by 41.50 and 56.83 percentage points, respectively, compared with using the same retrieval pipeline without altitude normalization, and the main retrieval pipeline runs at 13.3 frames/s on the workstation used for evaluation. These results indicate that relative altitude estimation provides a useful scale prior for cross-altitude UAV-to-map image matching using georeferenced remote sensing imagery.
Indian Institute of Technology, Kanpur（印度理工学院坎普尔分校）
In this work, we present a pragmatic approach to enable unmanned aerial vehicle (UAVs) to autonomously perform highly complicated tasks of object pick and place. This paper is largely inspired by challenge-2 of MBZIRC 2020 and is primarily focused on the task of assembling large 3D structures in outdoors and GPS-denied environments. Primary contributions of this system are: (i) a novel computationally efficient deep learning based unified multi-task visual perception system for target localization, part segmentation, and tracking, (ii) a novel deep learning based grasp state estimation, (iii) …
Institute for Systems Analysis of Russian Academy of Sciences（俄罗斯科学院系统分析研究所）；Soloviev Rybinsk State Aviation Technical University（索洛维约夫雷宾斯克国立航空技术大学）；NPP SATEK plus（SATEK plus公司）
The problem of developing distributed control and navigation system for quadrotor UAVs operating in GPS-denied environments is addressed in the paper. Cooperative navigation, marker detection and mapping task solved by a team of multiple unmanned aerial vehicles is chosen as demo example. Developed intelligent control system complies with on 4D\RCS reference model and its implementation is based on ROS framework. Custom implementation of EKF-based map building algorithm is used to solve marker detection and map building task.
Localization is one of the most important technologies needed to use Unmanned Aerial Vehicles (UAVs) in actual fields. Currently, most UAVs use GNSS to estimate their position. Recently, there have been attacks that target the weaknesses of UAVs that use GNSS, such as interrupting GNSS signal to crash the UAVs or sending fake GNSS signals to hijack the UAVs. To avoid this kind of situation, this paper proposes an algorithm that deals with the localization problem of the UAV in GNSS-denied environments. We propose a localization method, named as BRM (Building Ratio Map based) localization, for …
Queensland University of Technology（昆士兰科技大学）；Data61 / CSIRO（Data61 / 联邦科学与工业研究组织）；Griffith University（格里菲斯大学）
The use of multi-rotor Unmanned Aerial Vehicles (UAVs) for search and rescue as well as remote sensing is rapidly increasing. Multi-rotor UAVs, however, have limited endurance. The range of UAV applications can be widened if teams of multiple UAVs are used. We propose a framework for a team of UAVs to cooperatively explore and find a target in complex GPS-denied environments with obstacles. The team of UAVs autonomously navigates, explores, detects, and finds the target in a cluttered environment with a known map. Examples of such environments include indoor scenarios, urban or natural canyons…
Unmanned Aerial Vehicles (UAVs) have recently shown great performance collecting visual data through autonomous exploration and mapping in building inspection. Yet, the number of studies is limited considering the post processing of the data and its integration with autonomous UAVs. These will enable huge steps onward into full automation of building inspection. In this regard, this work presents a decision making tool for revisiting tasks in visual building inspection by autonomous UAVs. The tool is an implementation of fine-tuning a pretrained Convolutional Neural Network (CNN) for surface c…
UAVs have been widely used in visual inspections of buildings, bridges and other structures. In either outdoor autonomous or semi-autonomous flights missions strong GPS signal is vital for UAV to locate its own positions. However, strong GPS signal is not always available, and it can degrade or fully loss underneath large structures or close to power lines, which can cause serious control issues or even UAV crashes. Such limitations highly restricted the applications of UAV as a routine inspection tool in various domains. In this paper a vision-model-based real-time self-positioning method is …
Optimizing Indoor Farm Monitoring Efficiency Using UAV: Yield Estimation in a GNSS-Denied Cherry Tomato Greenhouse
Department of Mechanical Engineering, Ulsan National Institute of Science and Technology (UNIST)（机械工程系，乌山国立科学技术研究院）；Agricultural Robotics and Artificial Intelligence, Metafarmers（农业机器人与人工智能，Metafarmers）
University of Turku（图尔库大学）；École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院（EPFL））
Small unmanned aerial vehicles (UAV) have penetrated multiple domains over the past years. In GNSS-denied or indoor environments, aerial robots require a robust and stable localization system, often with external feedback, in order to fly safely. Motion capture systems are typically utilized indoors when accurate localization is needed. However, these systems are expensive and most require a fixed setup. Recently, visual-inertial odometry and similar methods have advanced to a point where autonomous UAVs can rely on them for localization. The main limitation in this case comes from the environ…
Unmanned aerial vehicles (UAVs) are becoming largely ubiquitous with an increasing demand for aerial data. Accurate navigation and localization, required for precise data collection in many industrial applications, often relies on RTK GNSS. These systems, able of centimeter-level accuracy, require a setup and calibration process and are relatively expensive. This paper addresses the problem of accurate positioning and navigation of UAVs through cooperative localization. Inexpensive ultra-wideband (UWB) transceivers installed on both the UAV and a support ground robot enable centimeter-level re…
This paper addresses the problem of position estimation in UAVs operating in a cluttered environment where GPS information is unavailable. A model learning-based approach is proposed that takes in the rotor RPMs and past state as input and predicts the one-step-ahead position of the UAV using a novel spectral-normalized memory neural network (SN-MNN). The spectral normalization guarantees stable and reliable prediction performance. The predicted position is transformed to global coordinate frame which is then fused along with the odometry of other peripheral sensors like IMU, barometer, compas…
We study the problem of infrastructure inspection using an Unmanned Aerial Vehicle (UAV) in box girder bridge environments. We consider a scenario where the UAV needs to fully inspect box girder bridges and localize along the bridge surface when standard methods like GPS and optical flow are denied. Our method for overcoming the difficulties of box girder bridges consist of creating local navigation routines, a supervisor, and a planner. The local navigation routines use two 2D Lidars for girder and column flight. For switching between local navigation routines we implement a supervisor which …
Drones Guiding Drones: Cooperative Navigation of a Less-Equipped Micro Aerial Vehicle in Cluttered Environments
Multi-robot Systems Group, Department of Cybernetics, Faculty of Electrical Engineering, Czech Technical University in Prague（多机器人系统组，控制系，电气工程系，捷克技术大学）
Cross-Session 3D LiDAR and Camera Fusion for Robust Localization of Unmanned Aerial Vehicles in GPS-Denied Environments
School of Electrical and Data Engineering, University of Technology Sydney（悉尼大学电气与数据工程学院）；Institute of Aerospace Engineering and Technology, Duy Tan University（杜益大学航空航天工程与技术学院）；College of Engineering and Computer Science, VinUniversity（Vin大学工程与计算机科学学院）；Smart Green Transformation Center (GREEN-X), VinUniversity（Vin大学智能绿色转型中心 (GREEN-X)）；Department of Artificial Intelligence and Robotics, Sejong University（成均馆大学人工智能与机器人系）
SOAR: Regression-based LiDAR Relocalization for UAVs
Fujian Key Laboratory of Sensing and Computing for Smart Cities, Xiamen University（厦门大学智慧城市感知与计算重点实验室）；Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University（中国教育部多媒体可信感知与高效计算重点实验室）；Department of Computer Science at the University of Oxford（牛津大学计算机科学系）
This paper proposes and benchmarks two tether-based motion primitives for tethered UAVs to execute autonomous flight with proprioception only. Tethered UAVs have been studied mainly due to power and safety considerations. Tether is either not included in the UAV motion (treated same as free-flying UAV) or only in terms of station-keeping and high-speed steady flight. However, feedback from and control over the tether configuration could be utilized as a set of navigational tools for autonomous flight, especially in GPS-denied environments and without vision-based exteroception. In this work, t…
We propose a framework for Google Map aided UAV navigation in GPS-denied environment. Geo-referenced navigation provides drift-free localization and does not require loop closures. The UAV position is initialized via correlation, which is simple and efficient. We then use optical flow to predict its position in subsequent frames. During pose tracking, we obtain inter-frame translation either by motion field or homography decomposition, and we use HOG features for registration on Google Map. We employ particle filter to conduct a coarse to fine search to localize the UAV. Offline test using aer…
The capabilities of autonomous flight with unmanned aerial vehicles (UAVs) have significantly increased in recent times. However, basic problems such as fast and robust geo-localization in GPS-denied environments still remain unsolved. Existing research has primarily concentrated on improving the accuracy of localization at the cost of long and varying computation time in various situations, which often necessitates the use of powerful ground station machines. In order to make image-based geo-localization online and pragmatic for lightweight embedded systems on UAVs, we propose a framework tha…
Communication-Free Collective Navigation for a Swarm of UAVs via LiDAR-Based Deep Reinforcement Learning
Department of Mechanical Engineering, Ulsan National Institute of Science and Technology（机械工程系，釜山国立科学技术研究院）；Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology（机械工程系，韩国科学技术院）
This paper presents fast procedures for thermal infrared remote sensing in dark, GPS-denied environments, such as those found in industrial plants such as in High-Voltage Direct Current (HVDC) converter stations. These procedures are based on the combination of the depth estimation obtained from either a 1-Dimensional LIDAR laser or a 2-Dimensional Hokuyo laser or a 3D MultiSense SLB laser sensor and the visible and thermal cameras from a FLIR Duo R dual-sensor thermal camera. The combination of these sensors/cameras is suitable to be mounted on Unmanned Aerial Vehicles (UAVs) and/or robots in…
Jet Propulsion Laboratory, California Institute of Technology（加州理工学院喷气推进实验室）
We propose TRADE for robust tracking and 3D localization of a moving target in cluttered environments, from UAVs equipped with a single camera. Ultimately TRADE enables 3d-aware target following. Tracking-by-detection approaches are vulnerable to target switching, especially between similar objects. Thus, TRADE predicts and incorporates the target 3D trajectory to select the right target from the tracker's response map. Unlike static environments, depth estimation of a moving target from a single camera is a ill-posed problem. Therefore we propose a novel 3D localization method for ground targ…
Deloitte LLP（德勤有限责任公司）；Royal Holloway University of London（伦敦大学皇家霍洛威学院）
The anticipated widespread use of unmanned aerial vehicles (UAVs) raises significant safety and security concerns, including trespassing in restricted areas, colliding with other UAVs, and disrupting high-traffic airspaces. To mitigate these risks, geofences have been proposed as one line of defence, which limit UAVs from flying into the perimeters of other UAVs and restricted locations. In this paper, we address the concern that existing geometric geofencing algorithms lack accuracy during the calculation of complex geofences, particularly in dynamic urban environments. We propose a new algor…
Considering the accelerated development of Unmanned Aerial Vehicles (UAVs) applications in both industrial and research scenarios, there is an increasing need for localizing these aerial systems in non-urban environments, using GNSS-Free, vision-based methods. Our paper proposes a vision-based localization algorithm that utilizes deep features to compute geographical coordinates of a UAV flying in the wild. The method is based on matching salient features of RGB photographs captured by the drone camera and sections of a pre-built map consisting of georeferenced open-source satellite images. Ex…
Virginia Tech（弗吉尼亚理工大学）；University of Maryland（马里兰大学）；University of Genoa（热那亚大学）
We develop an effective computer model to simulate sensing environments that consist of natural trees. The simulated environments are random and contain full geometry of the tree foliage. While this simulated model can be used as a general platform for studying the sensing mechanism of different flying species, our ultimate goal is to build bat-inspired Quad-rotor UAVs- UAVs that can recreate bat's flying behavior (e.g., obstacle avoidance, path planning) in dense vegetation. To this end, we also introduce an foliage echo simulator that can produce simulated echoes by mimicking bat's biosonar.…
Semantic-Aware Autonomous Exploration for UAVs in Unknown Indoor Environments
University of Engineering and Technology, Vietnam National University（越南国立大学工程与技术大学）；School of Information Science, Japan Advanced Institute of Science and Technology（日本先端科学技术大学院大学信息科学学院）
Precise geolocalization is crucial for unmanned aerial vehicles (UAVs). However, most current deployed UAVs rely on the global navigation satellite systems (GNSS) or high precision inertial navigation systems (INS) for geolocalization. In this paper, we propose to use a lightweight visual-inertial system with a 2D georeference map to obtain accurate and consecutive geodetic positions for UAVs. The proposed system firstly integrates a micro inertial measurement unit (MIMU) and a monocular camera as odometry to consecutively estimate the navigation states and reconstruct the 3D position of the o…
Autonomous UAV-Quadruped Docking in Complex Terrains via Active Posture Alignment and Constraint-Aware Control
College of Electronics and Information Engineering, Tongji University（同济大学电子与信息工程学院）；Frontiers Science Center for Intelligent Autonomous Systems, Shanghai（上海智能自主系统前沿科学中心）；School of Logistics Engineering, Shanghai Maritime University（上海 Maritime University物流工程学院）；State Key Laboratory of Autonomous Intelligent Unmanned Systems, Shanghai（上海自主智能无人系统国家重点实验室）
Åbo Akademi University（奥布学术大学）；KTH Royal Institute of Technology（瑞典皇家理工学院）
With the growing popularity of Unmanned Aerial Vehicles (UAVs) for consumer applications, the number of accidents involving UAVs is also increasing rapidly. Therefore, motion safety of UAVs has become a prime concern for UAV operators. For a swarm of UAVs, a safe operation can not be guaranteed without preventing the UAVs from colliding with one another and with static and dynamically appearing, moving obstacles in the flying zone. In this paper, we present an online, collision-free path generation and navigation system for swarms of UAVs. The proposed system uses geographical locations of the…
Solar power models are a crucial element of solar-powered UAV design and performance analysis. During the conceptual design phase, their accuracy directly relates to the accuracy of the predicted performance metrics and thus the final design characteristics of the solar-powered UAV. Likewise, during the operations phase of a solar-powered UAV accurate solar power income models are required to predict and assess the solar power system performance. However, the existing literature on solar-powered UAVs uses highly simplified solar power models. This technical report therefore, first, introduces …
Non-Contact Vibration-Based Damage Detection of Civil Structures Using a Cost-Effective Autonomous UAV
Department of Computer Science, The University of Texas at Rio Grande Valley (UTRGV)（德克萨斯大学里奥格兰德谷大学计算机科学系）；Department of Mechanical Engineering at UTRGV（德克萨斯大学里奥格兰德谷大学机械工程系）；Department of Civil Engineering at UTRGV（德克萨斯大学里奥格兰德谷大学土木工程系）；Department of Electrical and Computer Engineering at UTRGV（德克萨斯大学里奥格兰德谷大学电气与计算机工程系）
Neural-Geometric Tunnel Traversal: Localization-free UAV Flight with Tilted LiDARs
Centro Universitario de la Defensa, Zaragoza（萨拉戈萨国防大学中心）；Instituto de Investigación en Ingeniería de Aragón（阿拉贡工程研究院）；University of Zaragoza（萨拉戈萨大学）；Investigación en Ingeniería de Aragón（阿拉贡工程研究院）
