Visual Place Recognition for Aerial Robotics: Exploring Accuracy-Computation Trade-off for Local Image Descriptors
University of Essex（埃塞克斯大学）；COMSATS University Islamabad（伊斯兰堡COMSATS大学）；National University of Sciences and Technology (NUST)（国立科技大学（NUST））；Queensland University of Technology（昆士兰科技大学）
Bruno Ferrarini, Maria Waheed, Sania Waheed, Shoaib Ehsan, Michael Milford, Klaus D. McDonald-Maier
发表信息：
NASA/ESA Conference on Adaptive Hardware and Systems (AHS 2019)
Visual Place Recognition (VPR) is a fundamental yet challenging task for small Unmanned Aerial Vehicle (UAV). The core reasons are the extreme viewpoint changes, and limited computational power onboard a UAV which restricts the applicability of robust but computation intensive state-of-the-art VPR methods. In this context, a viable approach is to use local image descriptors for performing VPR as these can be computed relatively efficiently without the need of any special hardware, such as a GPU. However, the choice of a local feature descriptor is not trivial and calls for a detailed investigation as there is a trade-off between VPR accuracy and the required computational effort. To fill this research gap, this paper examines the performance of several state-of-the-art local feature descriptors, both from accuracy and computational perspectives, specifically for VPR application utilizing standard aerial datasets. The presented results confirm that a trade-off between accuracy and computational effort is inevitable while executing VPR on resource-constrained hardware.
This work addresses cross-view camera pose estimation, i.e., determining the 3-Degrees-of-Freedom camera pose of a given ground-level image w.r.t. an aerial image of the local area. We propose SliceMatch, which consists of ground and aerial feature extractors, feature aggregators, and a pose predictor. The feature extractors extract dense features from the ground and aerial images. Given a set of candidate camera poses, the feature aggregators construct a single ground descriptor and a set of pose-dependent aerial descriptors. Notably, our novel aerial feature aggregator has a cross-view atten…
Aerial scenes are more complicated in terms of object distribution and spatial arrangement than natural scenes due to the bird view, and thus remain challenging to learn discriminative scene representation. Recent solutions design \textit{local semantic descriptors} so that region of interests (RoIs) can be properly highlighted. However, each local descriptor has limited description capability and the overall scene representation remains to be refined. In this paper, we solve this problem by designing a novel representation set named \textit{instance representation bank} (IRB), which unifies m…
Exploring Image Representation with Decoupled Classical Visual Descriptors
The MIx Group University of Birmingham, UK（米克斯小组英国伯明翰大学）；University of Cambridge Cambridge, UK（剑桥大学）；Allsee Technologies Ltd Birmingham, UK（Allsee技术有限公司）
Tracking many vehicles in wide coverage aerial imagery is crucial for understanding events in a large field of view. Most approaches aim to associate detections from frame differencing into tracks. However, slow or stopped vehicles result in long-term missing detections and further cause tracking discontinuities. Relying merely on appearance clue to recover missing detections is difficult as targets are extremely small and in grayscale. In this paper, we address the limitations of detection association methods by coupling it with a local context tracker (LCT), which does not rely on motion det…
Integration of aerial and ground images has been proved as an efficient approach to enhance the surface reconstruction in urban environments. However, as the first step, the feature point matching between aerial and ground images is remarkably difficult, due to the large differences in viewpoint and illumination conditions. Previous studies based on geometry-aware image rectification have alleviated this problem, but the performance and convenience of this strategy is limited by several flaws, e.g. quadratic image pairs, segregated extraction of descriptors and occlusions. To address these pro…
German Aerospace Center (DLR)（德国航空航天中心）；Technical University of Munich (TUM)（慕尼黑工业大学）
Aerial image classification is of great significance in remote sensing community, and many researches have been conducted over the past few years. Among these studies, most of them focus on categorizing an image into one semantic label, while in the real world, an aerial image is often associated with multiple labels, e.g., multiple object-level labels in our case. Besides, a comprehensive picture of present objects in a given high resolution aerial image can provide more in-depth understanding of the studied region. For these reasons, aerial image multi-label classification has been attractin…
Autonomous Robots Lab, University of Nevada, Reno（内华达大学雷诺分校自主机器人实验室）
This paper proposes a method for tight fusion of visual, depth and inertial data in order to extend robotic capabilities for navigation in GPS-denied, poorly illuminated, and texture-less environments. Visual and depth information are fused at the feature detection and descriptor extraction levels to augment one sensing modality with the other. These multimodal features are then further integrated with inertial sensor cues using an extended Kalman filter to estimate the robot pose, sensor bias terms, and landmark positions simultaneously as part of the filter state. As demonstrated through a s…
Utrecht University（乌得勒支大学）；University of Amsterdam（阿姆斯特丹大学）；SIM University（SIM大学）
In this work, we aim to segment and detect water in videos. Water detection is beneficial for appllications such as video search, outdoor surveillance, and systems such as unmanned ground vehicles and unmanned aerial vehicles. The specific problem, however, is less discussed compared to general texture recognition. Here, we analyze several motion properties of water. First, we describe a video pre-processing step, to increase invariance against water reflections and water colours. Second, we investigate the temporal and spatial properties of water and derive corresponding local descriptors. Th…
Co-Registration of aerial imagery and Light Detection and Ranging (LiDAR) data is quilt challenging because the different imaging mechanism causes significant geometric and radiometric distortions between such data. To tackle the problem, this paper proposes an automatic registration method based on structural features and three-dimension (3D) phase correlation. In the proposed method, the LiDAR point cloud data is first transformed into the intensity map, which is used as the reference image. Then, we employ the Fast operator to extract uniformly distributed interest points in the aerial imag…
This paper investigates classification of traffic scenes in a very low bandwidth scenario, where an image should be coded by a small number of features. We introduce a novel dataset, called the FM1 dataset, consisting of 5615 images of eight different traffic scenes: open highway, open road, settlement, tunnel, tunnel exit, toll booth, heavy traffic and the overpass. We evaluate the suitability of the GIST descriptor as a representation of these images, first by exploring the descriptor space using PCA and k-means clustering, and then by using an SVM classifier and recording its 10-fold cross-…
The visual entities in cross-view images exhibit drastic domain changes due to the difference in viewpoints each set of images is captured from. Existing state-of-the-art methods address the problem by learning view-invariant descriptors for the images. We propose a novel method for solving this task by exploiting the generative powers of conditional GANs to synthesize an aerial representation of a ground level panorama and use it to minimize the domain gap between the two views. The synthesized image being from the same view as the target image helps the network to preserve important cues in …
The capabilities of autonomous flight with unmanned aerial vehicles (UAVs) have significantly increased in recent times. However, basic problems such as fast and robust geo-localization in GPS-denied environments still remain unsolved. Existing research has primarily concentrated on improving the accuracy of localization at the cost of long and varying computation time in various situations, which often necessitates the use of powerful ground station machines. In order to make image-based geo-localization online and pragmatic for lightweight embedded systems on UAVs, we propose a framework tha…
University of Bonn（波恩大学）；Fraunhofer IAIS（弗劳恩霍夫智能分析和信息系统研究所）；Bonn-Aachen International Center for Information Technology（波恩-亚琛信息技术国际中心）
Motivated by the center-surround mechanism in the human visual attention system, we propose to use average contrast maps for the challenge of pedestrian detection in street scenes due to the observation that pedestrians indeed exhibit discriminative contrast texture. Our main contributions are first to design a local, statistical multi-channel descriptorin order to incorporate both color and gradient information. Second, we introduce a multi-direction and multi-scale contrast scheme based on grid-cells in order to integrate expressive local variations. Contributing to the issue of selecting mo…
City University of Hong Kong（香港城市大学）；Zhejiang University of Technology（浙江工业大学）
Motion behaviors of a rigid body can be characterized by a 6-dimensional motion trajectory, which contains position vectors of a reference point on the rigid body and rotations of this rigid body over time. This paper devises a Rotation and Relative Velocity (RRV) descriptor by exploring the local translational and rotational invariants of motion trajectories of rigid bodies, which is insensitive to noise, invariant to rigid transformation and scaling. A flexible metric is also introduced to measure the distance between two RRV descriptors. The RRV descriptor is then applied to characterize mo…
Efficient Match Pair Retrieval for Large-scale UAV Images via Graph Indexed Global Descriptor
China University of Geosciences（中国地质大学）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (Shenzhen)（广东省人工智能与数字经济实验室（深圳））；Shenzhen University（深圳大学）；Wuhan University（武汉大学）；Hunan University of Science and Technology（湖南科技大学）
Facebook AI Research（Facebook AI研究院）；Carnegie Mellon University（卡内基梅隆大学）；University of São Paulo（圣保罗大学）
Aerial vehicles are revolutionizing the way film-makers can capture shots of actors by composing novel aerial and dynamic viewpoints. However, despite great advancements in autonomous flight technology, generating expressive camera behaviors is still a challenge and requires non-technical users to edit a large number of unintuitive control parameters. In this work, we develop a data-driven framework that enables editing of these complex camera positioning parameters in a semantic space (e.g. calm, enjoyable, establishing). First, we generate a database of video clips with a diverse range of sh…
Unsupervised Part Discovery via Descriptor-Based Masked Image Restoration with Optimized Constraints
Faculty of Engineering and IT, University of Technology Sydney（技术悉尼大学工程与信息学院）；Dept. of Comp. Sci. and Eng., Southern University of Science and Technology（南方科技大学计算机科学与工程系）
University of Sao Paulo（圣保罗大学）；University of Pennsylvania（宾夕法尼亚大学）；Treeswift
In this letter, we present a novel descriptor based on Urquhart tessellations derived from the position of trees in a forest. We propose a framework that uses these descriptors to detect previously seen observations and landmark correspondences, even with partial overlap and noise. We run loop closure detection experiments in simulation and real-world data map-merging from different flights of an Unmanned Aerial Vehicle (UAV) in a pine tree forest and show that our method outperforms state-of-the-art approaches in accuracy and robustness.
UAV-Rain1k: A Benchmark for Raindrop Removal from UAV Aerial Imagery
Shenyang Aerospace University（沈阳航空航天大学）；Naval Aviation University（海军航空大学）；Nanjing University of Science and Technology（南京理工大学）；Zhejiang Ocean University（浙江海洋大学）
Australian National University（澳大利亚国立大学）；Australian Centre for Robotic Vision（澳大利亚机器人视觉中心）；University of Technology Sydney（悉尼科技大学）
Cross-view geo-localization is the problem of estimating the position and orientation (latitude, longitude and azimuth angle) of a camera at ground level given a large-scale database of geo-tagged aerial (e.g., satellite) images. Existing approaches treat the task as a pure location estimation problem by learning discriminative feature descriptors, but neglect orientation alignment. It is well-recognized that knowing the orientation between ground and aerial images can significantly reduce matching ambiguity between these two views, especially when the ground-level images have a limited Field …
Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences（中国科学院西安光机精密机械研究所）；The University of Chinese Academy of Sciences（中国科学院大学）
Inspired by recent development of artificial satellite, remote sensing images have attracted extensive attention. Recently, noticeable progress has been made in scene classification and target detection.However, it is still not clear how to describe the remote sensing image content with accurate and concise sentences. In this paper, we investigate to describe the remote sensing images with accurate and flexible sentences. First, some annotated instructions are presented to better describe the remote sensing images considering the special characteristics of remote sensing images. Second, in ord…
Oblique images are aerial photographs taken at oblique angles to the earth's surface. Projections of vector and other geospatial data in these images depend on camera parameters, positions of the geospatial entities, surface terrain, occlusions, and visibility. This paper presents a robust and scalable algorithm to detect inconsistencies in vector data using oblique images. The algorithm uses image descriptors to encode the local appearance of a geospatial entity in images. These image descriptors combine color, pixel-intensity gradients, texture, and steerable filter responses. A Support Vect…
Exploring the added value of pretherapeutic MR descriptors in predicting breast cancer pathologic complete response to neoadjuvant chemotherapy
Institut Curie（居里研究所）；PSL Research University（巴黎文理研究大学）；Paris-Saclay University（巴黎-萨克雷大学）；Inserm（法国国家健康与医学研究院）；Gustave Roussy（古斯塔夫·鲁西研究所）；Centre Léon Bérard（莱昂·贝拉尔中心）；HCL（里昂民用医院）；University of Paris（巴黎大学）
We present a real-time approach for image-based localization within large scenes that have been reconstructed offline using structure from motion (Sfm). From monocular video, our method continuously computes a precise 6-DOF camera pose, by efficiently tracking natural features and matching them to 3D points in the Sfm point cloud. Our main contribution lies in efficiently interleaving a fast keypoint tracker that uses inexpensive binary feature descriptors with a new approach for direct 2D-to-3D matching. The 2D-to-3D matching avoids the need for online extraction of scale-invariant features. …
Geometry-Aware Cross-Height Channel Knowledge Map Prediction for UAV-Assisted Communications With Uncertainty-Guided 3D Sensing
National Key Laboratory of Wireless Communications, University of Electronic Science and Technology of China (UESTC)（电子科技大学无线通信国家重点实验室）；SDAIA-KFUPM Joint Research Centre of AI, King Fahd University of Petroleum and Minerals（沙特数据和人工智能局-法赫德国王石油矿产大学人工智能联合研究中心）；School of Science, Technology and Engineering, University of the Sunshine Coast（阳光海岸大学科学技术与工程学院）；Anhui Science and Technology University（安徽科技学院）；College of Telecommunications and Information Engineering, Nanjing University of Posts and Telecommunications（南京邮电大学通信与信息工程学院）
SceneMixer: Exploring Convolutional Mixing Networks for Remote Sensing Scene Classification
College of Engineering（工程学院）；University of Dubai（迪拜大学）；Department of Geography（地理系）；Simon Fraser University（西蒙弗雷泽大学）；Department of Computer Science（计算机科学系）；Tezpur University（泰朱普大学）
Institute of Engineering & Management（工程与管理学院）；Indian Institute of Technology Roorkee（鲁尔基印度理工学院）
In this paper, a new texture descriptor named "Fractional Local Neighborhood Intensity Pattern" (FLNIP) has been proposed for content based image retrieval (CBIR). It is an extension of the Local Neighborhood Intensity Pattern (LNIP)[1]. FLNIP calculates the relative intensity difference between a particular pixel and the center pixel of a 3x3 window by considering the relationship with adjacent neighbors. In this work, the fractional change in the local neighborhood involving the adjacent neighbors has been calculated first with respect to one of the eight neighbors of the center pixel of a 3…
Modality-Aware Feature Matching in Visual and Vision-Language Applications: A Comprehensive Survey
School of Computing and Artificial Intelligence, Jiangxi University of Finance and Economics（江西财经大学计算机与人工智能学院）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算机与数据科学学院）；School of Computer Science and Informatics, Cardiff University（卡迪夫大学计算机科学与信息学院）；School of Computing and Communications, Lancaster University（兰卡斯特大学计算机与通讯学院）；School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；Institute for Infocomm Research, Agency for Science, Technology and Research (A*STAR)（新加坡资讯研究院，科技研究局（A*STAR））；Department of Automation, Tsinghua University（清华大学自动化系）
Universidad Politécnica de Madrid（马德里理工大学）；Centre for Automation and Robotics (CAR UPM-CSIC)（自动化与机器人中心（马德里理工大学-西班牙国家研究委员会））；ETSIDI（马德里理工大学工业设计工程学院）
The use of 3D cameras for gait analysis has been highly questioned due to the low accuracy they have demonstrated in the past. The objective of the study presented in this paper is to improve the accuracy of the estimations made by robot-mounted 3D cameras in human gait analysis by applying a supervised learning stage. The 3D camera was mounted in a mobile robot to obtain a longer walking distance. This study shows an improvement in detection of kinematic gait signals and gait descriptors by post-processing the raw estimations of the camera using artificial neural networks trained with the dat…
Beijing University of Posts and Telecommunications（北京邮电大学）；Megvii Co.（旷视科技有限公司）
As the foundation of driverless vehicle and intelligent robots, Simultaneous Localization and Mapping(SLAM) has attracted much attention these days. However, non-geometric modules of traditional SLAM algorithms are limited by data association tasks and have become a bottleneck preventing the development of SLAM. To deal with such problems, many researchers seek to Deep Learning for help. But most of these studies are limited to virtual datasets or specific environments, and even sacrifice efficiency for accuracy. Thus, they are not practical enough. We propose DF-SLAM system that uses deep loc…
Southwest Jiaotong University（西南交通大学）；The Chinese University of Hong Kong（香港中文大学）；Nanyang Technological University（南洋理工大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
Video based action recognition is one of the important and challenging problems in computer vision research. Bag of Visual Words model (BoVW) with local features has become the most popular method and obtained the state-of-the-art performance on several realistic datasets, such as the HMDB51, UCF50, and UCF101. BoVW is a general pipeline to construct a global representation from a set of local features, which is mainly composed of five steps: (i) feature extraction, (ii) feature pre-processing, (iii) codebook generation, (iv) feature encoding, and (v) pooling and normalization. Many efforts ha…
ReLATE: Reliability-Guided Evidence Fusion for Robust UAV--Satellite cross-view Geo-Localization
School of Electronics and Information Engineering, Harbin Institute of Technology（哈尔滨工业大学电子与信息工程学院）；National Key Laboratory of Radar Detection and Sensing, Nanjing Research Institute of Electronics Technology（南京电子技术研究所雷达探测与感知国家重点实验室）；School of Electrical and Electronic Engineering, University of Manchester（曼彻斯特大学电气与电子工程学院）
OPAL: Visibility-aware LiDAR-to-OpenStreetMap Place Recognition via Adaptive Radial Fusion
Technical University of Munich（慕尼黑技术大学）；Wuhan University（武汉大学）；University of Science and Technology of China（中国科学技术大学）；University of Cambridge（剑桥大学）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
