Owing to the capacity of performing full-time target search, cross-modality vehicle re-identification (Re-ID) based on unmanned aerial vehicle (UAV) is gaining more attention in both video surveillance and public security. However, this promising and innovative research has not been studied sufficiently due to the data inadequacy issue. Meanwhile, the cross-modality discrepancy and orientation discrepancy challenges further aggravate the difficulty of this task. To this end, we pioneer a cross-modality vehicle Re-ID benchmark named UAV Cross-Modality Vehicle Re-ID (UCM-VeID), containing 753 identities with 16015 RGB and 13913 infrared images. Moreover, to meet cross-modality discrepancy and orientation discrepancy challenges, we present a hybrid weights decoupling network (HWDNet) to learn the shared discriminative orientation-invariant features. For the first challenge, we proposed a hybrid weights siamese network with a well-designed weight restrainer and its corresponding objective function to learn both modality-specific and modality shared information. In terms of the second challenge, three effective decoupling structures with two pretext tasks are investigated to learn orientation-invariant feature. Comprehensive experiments are carried out to validate the effectiveness of the proposed method. The dataset and codes will be released at https://github.com/moonstarL/UAV-CM-VeID.
Feature-Augmented Deep Networks for Multiscale Building Segmentation in High-Resolution UAV and Satellite Imagery
Center for Geospatial Research, Department of Geography University of Georgia（地理空间研究中心，地理系，佐治亚大学）；Photogrammetry and Remote Sensing Department Indian Institute of Remote Sensing（摄影测量与遥感系，印度遥感研究所）；SEAI Lab, Department of Geography and the Environment University of Texas at Austin（SEAI实验室，地理与环境系，德克萨斯大学奥斯汀分校）
UAV-CB: A Complex-Background RGB-T Dataset and Local Frequency Bridge Network for UAV Detection
Pengcheng Laboratory（鹏城实验室）；South China University of Technology（南方科技大学）；Peking University（北京大学）；Xinjiang University（新疆大学）；Harbin Institute of Technology（哈尔滨工业大学）
Unmanned Aerial Vehicles (UAVs) have become popular for use in plant phenotyping of field based crops, such as maize and sorghum, due to their ability to acquire high resolution data over field trials. Field experiments, which may comprise thousands of plants, are planted according to experimental designs to evaluate varieties or management practices. For many types of phenotyping analysis, we examine smaller groups of plants known as "plots." In this paper, we propose a new plot extraction method that will segment a UAV image into plots. We will demonstrate that our method achieves higher plo…
Evaluation of UAV-Based RGB and Multispectral Vegetation Indices for Precision Agriculture in Palm Tree Cultivation
College of Engineering and IT, University of Dubai（迪拜大学工程与信息技术学院）；Department of Electronics and Communication, TKM College of Engineering（TKM工程学院电子与通信系）；Department of Electronics and Communication, MES College of Engineering（MES工程学院电子与通信系）；Applications Development and Analysis Section, Mohammed Bin Rashid Space Centre (MBRSC)（穆罕默德·本·拉希德航天中心应用开发与分析部门）
Monitoring of reforestation is currently being considerably streamlined through the use of drones and image recognition algorithms, which have already proven to be effective on colour imagery. In addition to colour imagery, elevation data is often also available. The primary aim of this work was to improve the performance of the faster-RCNN object detection algorithm by integrating this height information, which showed itself to notably improve performance. Interestingly, the structure of the network played a key role, with direct addition of the height information as a fourth image channel sh…
Graph-based Semantic Calibration Network for Unaligned UAV RGBT Image Semantic Segmentation and A Large-scale Benchmark
Key Laboratory of Intelligent Computing & Signal Processing (Anhui University), Ministry of Education（智能计算与信号处理重点实验室（安徽大学），教育部）；Anhui Provincial Key Laboratory of Multimodal Cognitive Computation, School of Artificial Intelligence, Anhui University（安徽省多模态认知计算重点实验室，人工智能学院，安徽大学）；School of Computer Science and Technology, Anhui University（计算机科学与技术学院，安徽大学）；GEOVIS Earth Technology Co., Ltd.（GEOVIS地球科技有限公司）
Towards UAV Detection in the Real World: A New Multispectral Dataset UAVNet-MS and a New Method
College of Electronic Science and Technology, National University of Defense Technology（电子科学与技术学院，国防科技大学）；Aviation University of Air Force（空军航空大学）；Sun Yat-sen University（中山大学）
In many agricultural applications one wants to characterize physical properties of plants and use the measurements to predict, for example biomass and environmental influence. This process is known as phenotyping. Traditional collection of phenotypic information is labor-intensive and time-consuming. Use of imagery is becoming popular for phenotyping. In this paper, we present methods to estimate traits of sorghum plants from RBG cameras on board of an unmanned aerial vehicle (UAV). The position and orientation of the imagery together with the coordinates of sparse points along the area of int…
EGM-Det: Entropy-Guided Multimodal Adaptive Fusion for UAV RGB-IR Object Detection
School of Cybersecurity, Northwestern Polytechnical University（西北工业大学网络空间安全学院）；School of Automation and Software Engineering, Shanxi University（山西大学自动化与软件学院）
Dalian University of Technology（大连理工大学）；Peng Cheng Laboratory（鹏城实验室）；Tiwaki Co.Ltd.（Tiwaki有限公司）
With the popularity of multi-modal sensors, visible-thermal (RGB-T) object tracking is to achieve robust performance and wider application scenarios with the guidance of objects' temperature information. However, the lack of paired training samples is the main bottleneck for unlocking the power of RGB-T tracking. Since it is laborious to collect high-quality RGB-T sequences, recent benchmarks only provide test sequences. In this paper, we construct a large-scale benchmark with high diversity for visible-thermal UAV tracking (VTUAV), including 500 sequences with 1.7 million high-resolution (192…
Macau University of Science and Technology（澳门科技大学）；Alibaba Group（阿里巴巴集团）；National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所模式识别国家重点实验室）
Decoupling spatiotemporal representation refers to decomposing the spatial and temporal features into dimension-independent factors. Although previous RGB-D-based motion recognition methods have achieved promising performance through the tightly coupled multi-modal spatiotemporal representation, they still suffer from (i) optimization difficulty under small data setting due to the tightly spatiotemporal-entangled modeling;(ii) information redundancy as it usually contains lots of marginal information that is weakly relevant to classification; and (iii) low interaction between multi-modal spati…
Decoupling Ego-Motion from Target Dynamics via Dual-Interval Motion Cues for UAV Detection
Department of Robotics, School of Advanced Manufacturing and Robotics（机器人学院，先进制造与机器人学院）；State Key Laboratory of Turbulence and Complex Systems（湍流与复杂系统国家重点实验室）；Peking University（北京大学）；Great Bay University（大湾大学）
STATrack: A Target-Aware Fully Spiking Neural Network for Efficient UAV Tracking
College of Computer Science and Engineering, Guilin University of Technology（桂林理工大学计算机科学与工程学院）；School of Artificial Intelligence, Sun Yat-sen University（中山大学人工智能学院）；School of Electronic Information, Central South University（中南大学电子信息学院）
USU-Corn-WeedDB: A UAV RGB Image Dataset for Multi-Species Weed Detection in Forage Corn
Department of Applied Sciences, Technology, and Education, Utah State University（应用科学、技术和教育系，犹他州立大学）；Department of Plants, Soils & Climate, Utah State University（植物、土壤与气候系，犹他州立大学）；Department of Civil and Environmental Engineering, Utah State University（土木与环境工程系，犹他州立大学）
UAUTrack: Towards Unified Multimodal Anti-UAV Visual Tracking
College of Intelligent Robotics and Advanced Manufacturing（智能机器人与先进制造学院）；Fudan University（复旦大学）；School of Computer Science and Technology（计算机科学与技术学院）；Zhejiang Normal University（浙江师范大学）；Department of Mechanical Engineering（机械工程系）；The Hong Kong Polytechnic University（香港理工大学）
Method of UAV Inspection of Photovoltaic Modules Using Thermal and RGB Data Fusion
Khmelnytskyi National University（赫梅利尼茨基国立大学）；Department of Informatics and Teleinformatics（信息学与电信学系）；Kazimierz Pulaski University of Ra-dom（卡齐米日·波尼亚图夫斯基大学）；National Academy of the State Border Service of Ukraine named after Bogdan Khmelnitsky（以博丹·赫梅利尼茨基命名的乌克兰国家边境服务学院）
Decoupling Amplitude and Phase Attention in Frequency Domain for RGB-Event based Visual Object Tracking
School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院）；Northeastern University（东北大学）；Beijing Institute of Technology（北京理工大学）；School of Computer and Communication Engineering, University of Science and Technology Beijing（北京科技大学计算机与通信工程学院）；Peng Cheng Laboratory（鹏城实验室）；National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University（北京大学多媒体信息处理国家重点实验室）；School of Electronic and Computer Engineering, Shenzhen Graduate School, Peking University（北京大学深圳研究生院电子与计算机工程学院）
Hiphen SAS；INRAE（法国国家农业食品与环境研究院）；Arvalis；The University of Tokyo（东京大学）
Early-stage plant density is an essential trait that determines the fate of a genotype under given environmental conditions and management practices. The use of RGB images taken from UAVs may replace traditional visual counting in fields with improved throughput, accuracy and access to plant localization. However, high-resolution (HR) images are required to detect small plants present at early stages. This study explores the impact of image ground sampling distance (GSD) on the performances of maize plant detection at 3-5 leaves stage using Faster-RCNN. Data collected at HR (GSD=0.3cm) over 6 …
Cross-scene model adaption is crucial for camera relocalization in real scenarios. It is often preferable that a pre-learned model can be fast adapted to a novel scene with as few training samples as possible. The existing state-of-the-art approaches, however, can hardly support such few-shot scene adaption due to the entangling of image feature extraction and scene coordinate regression. To address this issue, we approach camera relocalization with a decoupled solution where feature extraction, coordinate regression, and pose estimation are performed separately. Our key insight is that featur…
FreeDriveRF: Monocular RGB Dynamic NeRF without Poses for Autonomous Driving via Point-Level Dynamic-Static Decoupling
Department of Automation, Key Laboratory of System Control and Information Processing of Ministry of Education, Key Laboratory of Marine Intelligent Equipment and System of Ministry of Education, Shanghai Engineering Research Center of Intelligent Control and Management（自动化系、教育部长系统控制与信息处理重点实验室、教育部长海洋智能设备与系统重点实验室、上海智能控制与管理工程研究中心）；Dimanshen Technology Co., Ltd.（迪马森科技有限公司）；Engineering Research Center of Intelligent Control for Underground Space, Ministry of Education, School of Information and Control Engineering, Advanced Robotics Research Center, China University of Mining and Technology（地下空间智能控制工程研究中心、教育部长、信息与控制工程学院、先进机器人研究中心、中国矿业大学）
EgoSIS: From Factorized Visual Ego-Transitions to Motion-Canonical Spatial Evidence for UAV Reasoning
Beihang University（北京航空航天大学）；Zhongguancun Academy（中关村学院）；Northeastern University（东北大学）；Technology and Engineering Center for Space Utilization, Chinese Academy of Sciences（中国科学院空间应用工程与技术中心）
Unmanned aerial vehicles (UAVs) equipped with multiple complementary sensors have tremendous potential for fast autonomous or remote-controlled semantic scene analysis, e.g., for disaster examination. In this work, we propose a UAV system for real-time semantic inference and fusion of multiple sensor modalities. Semantic segmentation of LiDAR scans and RGB images, as well as object detection on RGB and thermal images, run online onboard the UAV computer using lightweight CNN architectures and embedded inference accelerators. We follow a late fusion approach where semantic information from mult…
Singapore University of Technology and Design（新加坡科技设计大学）；Shandong University（山东大学）
Human behavior understanding with unmanned aerial vehicles (UAVs) is of great significance for a wide range of applications, which simultaneously brings an urgent demand of large, challenging, and comprehensive benchmarks for the development and evaluation of UAV-based models. However, existing benchmarks have limitations in terms of the amount of captured data, types of data modalities, categories of provided tasks, and diversities of subjects and environments. Here we propose a new benchmark - UAVHuman - for human behavior understanding with UAVs, which contains 67,428 multi-modal video sequ…
Unmanned aerial vehicles (UAVs) equipped with multiple complementary sensors have tremendous potential for fast autonomous or remote-controlled semantic scene analysis, e.g., for disaster examination. Here, we propose a UAV system for real-time semantic inference and fusion of multiple sensor modalities. Semantic segmentation of LiDAR scans and RGB images, as well as object detection on RGB and thermal images, run online onboard the UAV computer using lightweight CNN architectures and embedded inference accelerators. We follow a late fusion approach where semantic information from multiple sen…
National Institute of Technology, Rourkela（鲁尔凯拉国家理工学院）
Vision-based pose estimation of Unmanned Aerial Vehicles (UAV) in unknown environments is a rapidly growing research area in the field of robot vision. The task becomes more complex when the only available sensor is a static single camera (monocular vision). In this regard, we propose a monocular vision assisted localization algorithm, that will help a UAV to navigate safely in indoor corridor environments. Always, the aim is to navigate the UAV through a corridor in the forward direction by keeping it at the center with no orientation either to the left or right side. The algorithm makes use …
We present the ALTO dataset, a vision-focused dataset for the development and benchmarking of Visual Place Recognition and Localization methods for Unmanned Aerial Vehicles. The dataset is composed of two long (approximately 150km and 260km) trajectories flown by a helicopter over Ohio and Pennsylvania, and it includes high precision GPS-INS ground truth location data, high precision accelerometer readings, laser altimeter readings, and RGB downward facing camera imagery. In addition, we provide reference imagery over the flight paths, which makes this dataset suitable for VPR benchmarking and…
