Cross-view geo-localization is a task of matching the same geographic image from different views, e.g., unmanned aerial vehicle (UAV) and satellite. The most difficult challenges are the position shift and the uncertainty of distance and scale. Existing methods are mainly aimed at digging for more comprehensive fine-grained information. However, it underestimates the importance of extracting robust feature representation and the impact of feature alignment. The CNN-based methods have achieved great success in cross-view geo-localization. However it still has some limitations, e.g., it can only extract part of the information in the neighborhood and some scale reduction operations will make some fine-grained information lost. In particular, we introduce a simple and efficient transformer-based structure called Feature Segmentation and Region Alignment (FSRA) to enhance the model's ability to understand contextual information as well as to understand the distribution of instances. Without using additional supervisory information, FSRA divides regions based on the heat distribution of the transformer's feature map, and then aligns multiple specific regions in different views one on one. Finally, FSRA integrates each region into a set of feature representations. The difference is that FSRA does not divide regions manually, but automatically based on the heat distribution of the feature map. So that specific instances can still be divided and aligned when there are significant shifts and scale changes in the image. In addition, a multiple sampling strategy is proposed to overcome the disparity in the number of satellite images and that of images from other sources. Experiments show that the proposed method has superior performance and achieves the state-of-the-art in both tasks of drone view target localization and drone navigation. Code will be released at https://github.com/Dmmm1997/FSRA
Institute of Systems Science, National University of Singapore（新加坡国立大学系统科学研究所）
In this paper, we propose an novel implementation of a simultaneous localization and mapping (SLAM) system based on a monocular camera from an unmanned aerial vehicle (UAV) using Depth prediction performed with Capsule Networks (CapsNet), which possess improvements over the drawbacks of the more widely-used Convolutional Neural Networks (CNN). An Extended Kalman Filter will assist in estimating the position of the UAV so that we are able to update the belief for the environment. Results will be evaluated on a benchmark dataset to portray the accuracy of our intended approach.
ReLATE: Reliability-Guided Evidence Fusion for Robust UAV--Satellite cross-view Geo-Localization
School of Electronics and Information Engineering, Harbin Institute of Technology（哈尔滨工业大学电子与信息工程学院）；National Key Laboratory of Radar Detection and Sensing, Nanjing Research Institute of Electronics Technology（南京电子技术研究所雷达探测与感知国家重点实验室）；School of Electrical and Electronic Engineering, University of Manchester（曼彻斯特大学电气与电子工程学院）
UAV-VisLoc: A Large-scale Dataset for UAV Visual Localization
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；City University of Hong Kong（香港城市大学）；Beijing University of Posts and Telecommunications（北京邮电大学）
Jointly Optimized Global-Local Visual Localization of UAVs
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；City University of Hong Kong（香港城市大学）；Aerospace Information Research Institute, CAS（中国科学院空天信息创新研究院）；Beijing University of Posts and Telecommunications（北京邮电大学）
Research in the field of autonomous Unmanned Aerial Vehicles (UAVs) has significantly advanced in recent years, mainly due to their relevance in a large variety of commercial, industrial, and military applications. However, UAV navigation in GPS-denied environments continues to be a challenging problem that has been tackled in recent research through sensor-based approaches. This paper presents a novel offline, portable, real-time in-door UAV localization technique that relies on macro-feature detection and matching. The proposed system leverages the support of machine learning, traditional co…
Technology and Engineering Center for Space Utilization, Chinese Academy of Sciences（中国科学院空间应用工程与技术中心）；Chinese Academy of Sciences（中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）
The autonomous real-time optical navigation of planetary UAV is of the key technologies to ensure the success of the exploration. In such a GPS denied environment, vision-based localization is an optimal approach. In this paper, we proposed a multi-modal registration based SLAM algorithm, which estimates the location of a planet UAV using a nadir view camera on the UAV compared with pre-existing digital terrain model. To overcome the scale and appearance difference between on-board UAV images and pre-installed digital terrain model, a theoretical model is proposed to prove that topographic fea…
Beyond Matching to Tiles: Bridging Unaligned Aerial and Satellite Views for Vision-Only UAV Navigation
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；State Key Laboratory of Blockchain and Data Security, Zhejiang University（浙江大学区块链与数据安全全国重点实验室）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新区（滨江）区块链与数据安全研究院）
Trajectory Optimization in Single and Dual-UAV Bearing-Only Target Localization
College of Aerospace Science and Engineering, National University of Defense Technology（国防科技大学航天科学与工程学院）；Hunan Key Laboratory of Image Measurement and Visual Navigation（湖南省图像测量与视觉导航重点实验室）
National Institute of Technology, Rourkela（鲁尔凯拉国家理工学院）
Vision-based pose estimation of Unmanned Aerial Vehicles (UAV) in unknown environments is a rapidly growing research area in the field of robot vision. The task becomes more complex when the only available sensor is a static single camera (monocular vision). In this regard, we propose a monocular vision assisted localization algorithm, that will help a UAV to navigate safely in indoor corridor environments. Always, the aim is to navigate the UAV through a corridor in the forward direction by keeping it at the center with no orientation either to the left or right side. The algorithm makes use …
Scale-Aware UAV-to-Satellite Cross-View Geo-Localization: A Semantic Geometric Approach
College of Aerospace Science and Engineering, National University of Defense Technology（航天工程学院，国防科技大学）；Hunan Provincial Key Laboratory of Image Measurement and Vision Navigation（湖南省图像测量与视觉导航重点实验室）；School of Information Science and Engineering, Lanzhou University（信息科学与工程学院，兰州大学）
University of Toronto Institute for Aerospace Studies（多伦多大学航空航天研究所）
We propose and demonstrate a fast, robust method for using satellite images to localize an Unmanned Aerial Vehicle (UAV). Previous work using satellite images has large storage and computation costs and is unable to run in real time. In this work, we collect Google Earth (GE) images for a desired flight path offline and an autoencoder is trained to compress these images to a low-dimensional vector representation while retaining the key features. This trained autoencoder is used to compress a real UAV image, which is then compared to the precollected, nearby, autoencoded GE images using an inne…
Unifying UAV Cross-View Geo-Localization via 3D Geometric Perception
School of Electronic Information, Wuhan University（武汉大学电子信息学院）；School of Artificial Intelligence, Wuhan University（武汉大学人工智能学院）；Technology and Engineering Center for Space Utilization and the Key Laboratory of Space Utilization, Chinese Academy of Sciences（中国科学院空间应用工程与技术中心暨空间利用重点实验室）；School of Aeronautics and Astronautics, University of Chinese Academy of Sciences（中国科学院大学航空宇航学院）
This paper presents a framework for the localization of Unmanned Aerial Vehicles (UAVs) in unstructured environments with the help of deep learning. A real-time rendering engine is introduced that generates optical and depth images given a six Degrees-of-Freedom (DoF) camera pose, camera model, geo-referenced orthoimage, and elevation map. The rendering engine is embedded into a learning-based six-DoF Inverse Compositional Lucas-Kanade (ICLK) algorithm that is able to robustly align the rendered and real-world image taken by the UAV. To learn the alignment under environmental changes, the arch…
Structure-Aware Matching for Robust UAV Thermal-to-Satellite Geo-Localization
State Key Laboratory of Spatial Datum, Henan University（河南大学空间数据基准国家重点实验室）；College of Remote Sensing and Geoinformatics Engineering, Henan University（河南大学遥感与地理信息工程学院）；School of Computer and Information Engineering, Henan University（河南大学计算机与信息工程学院）；Henan Yunhuan NetLink UAV Technology Co., Ltd.（河南云环网联无人机科技有限公司）
PRI-Net: A Lightweight Multimodal Framework for 3D UAV Localization
College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息与电子工程学院）；Zhejiang Key Laboratory of Multimodal Communication Networks and Intelligent Information Processing（浙江省多模态通信网络与智能信息处理重点实验室）
Drone Referring Localization: An Efficient Heterogeneous Spatial Feature Interaction Method For UAV Self-Localization
School of Automation, Southeast University（东南大学自动化学院）；Unmanned System Application Technology Research Institute, China Jiliang University（中国计量大学无人系统应用技术研究院）；School of Artificial Intelligence and Computer Science, Jiangnan University（江南大学人工智能与计算机学院）；School of Computer Science, Southeast University（东南大学计算机科学学院）
We present the ALTO dataset, a vision-focused dataset for the development and benchmarking of Visual Place Recognition and Localization methods for Unmanned Aerial Vehicles. The dataset is composed of two long (approximately 150km and 260km) trajectories flown by a helicopter over Ohio and Pennsylvania, and it includes high precision GPS-INS ground truth location data, high precision accelerometer readings, laser altimeter readings, and RGB downward facing camera imagery. In addition, we provide reference imagery over the flight paths, which makes this dataset suitable for VPR benchmarking and…
DSM-Free UAV Visual Localization via Latent-Depth Variable Projection and Structure-Adaptive Coupling
College of Aerospace Science and Engineering, National University of Defense Technology（国防科技大学航天科学与工程学院）；College of Electronic Science and Technology, National University of Defense Technology（国防科技大学电子科学与工程学院）
Unmanned aerial vehicles (UAVs) have increasingly been adopted for safety, security, and rescue missions, for which they need precise and reliable pose estimates relative to their environment. To ensure mission safety when relying on visual perception, it is essential to have an approach to assess the integrity of the visual localization solution. However, to the best of our knowledge, such an approach does not exist for optimization-based visual localization. Receiver autonomous integrity monitoring (RAIM) has been widely used in global navigation satellite systems (GNSS) applications such as…
Nanyang Technological University（南洋理工大学）；School of Computer Science and Engineering, Northeastern University（东北大学计算机科学与工程学院）；Beihang University（北航大学）；University of Electronic Science and Technology of China（电子科技大学）
UAVs have been widely used in visual inspections of buildings, bridges and other structures. In either outdoor autonomous or semi-autonomous flights missions strong GPS signal is vital for UAV to locate its own positions. However, strong GPS signal is not always available, and it can degrade or fully loss underneath large structures or close to power lines, which can cause serious control issues or even UAV crashes. Such limitations highly restricted the applications of UAV as a routine inspection tool in various domains. In this paper a vision-model-based real-time self-positioning method is …
The paper focuses on the problem of vision-based obstacle detection and tracking for unmanned aerial vehicle navigation. A real-time object localization and tracking strategy from monocular image sequences is developed by effectively integrating the object detection and tracking into a dynamic Kalman model. At the detection stage, the object of interest is automatically detected and localized from a saliency map computed via the image background connectivity cue at each frame; at the tracking stage, a Kalman filter is employed to provide a coarse prediction of the object state, which is furthe…
We propose a framework for Google Map aided UAV navigation in GPS-denied environment. Geo-referenced navigation provides drift-free localization and does not require loop closures. The UAV position is initialized via correlation, which is simple and efficient. We then use optical flow to predict its position in subsequent frames. During pose tracking, we obtain inter-frame translation either by motion field or homography decomposition, and we use HOG features for registration on Google Map. We employ particle filter to conduct a coarse to fine search to localize the UAV. Offline test using aer…
SkyLink: Unifying Street-Satellite Geo-Localization via UAV-Mediated 3D Scene Alignment
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University（多媒体可信感知与高效计算重点实验室，教育部，厦门大学）；School of Electronic and Information Engineering, Foshan University（电子与信息工程学院，佛山大学）
OGMN: Occlusion-guided Multi-task Network for Object Detection in UAV Images
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Beijing Institute of Tracking and Telecommunication Technology（北京跟踪与通信技术研究所）
Scale-adaptive UAV Geo-localization via Height-aware Partition Learning
School of Automation, Hangzhou Dianzi University（杭州电子科技大学自动化学院）；School of Communication Engineering, Hangzhou Dianzi University（杭州电子科技大学通信工程学院）；Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）；Institute of Collaborative Innovation, University of Macau（澳门大学协同创新研究院）
Altitude-Adaptive Vision-Only Geo-Localization for UAVs in GPS-Denied Environments
Department of Precision Instrument, Tsinghua University（清华大学精密仪器系）；School of Aerospace Engineering, Beijing Institute of Technology（北京理工大学航天工程学院）；School of Instrumentation Science and Opto-electronics Engineering, Beijing Information Science and Technology University（北京信息科技大学仪器科学与光电工程学院）
