Precisely detection of Unmanned Aerial Vehicles(UAVs) plays a critical role in UAV defense systems. Deep learning is widely adopted for UAV object detection whereas researches on this topic are limited by the amount of dataset and small scale of UAV. To tackle these problems, a novel comprehensive approach that combines transfer learning based on simulation data and adaptive fusion is proposed. Firstly, the open-source plugin AirSim proposed by Microsoft is used to generate mass realistic simulation data. Secondly, transfer learning is applied to obtain a pre-trained YOLOv5 model on the simulated dataset and fine-tuned model on the real-world dataset. Finally, an adaptive fusion mechanism is proposed to further improve small object detection performance. Experiment results demonstrate the effectiveness of simulation-based transfer learning which leads to a 2.7% performance increase on UAV object detection. Furthermore, with transfer learning and adaptive fusion mechanism, 7.1% improvement is achieved compared to the original YOLO v5 model.
Cross-modal Offset-guided Dynamic Alignment and Fusion for Weakly Aligned UAV Object Detection
State Key Laboratory of Optical Field Manipulation Science and Technology, Chinese Academy of Sciences（光学场操控科学与技术国家重点实验室，中国科学院）；Institute of Optics and Electronics, Chinese Academy of Sciences（中国科学院光电研究所）
EGM-Det: Entropy-Guided Multimodal Adaptive Fusion for UAV RGB-IR Object Detection
School of Cybersecurity, Northwestern Polytechnical University（西北工业大学网络空间安全学院）；School of Automation and Software Engineering, Shanxi University（山西大学自动化与软件学院）
University of Maryland Baltimore County（马里兰大学巴尔的摩县分校）；Texas A&M University（德克萨斯A&M大学）
In this paper, we present a large-scale hurricane Michael dataset for visual perception in disaster scenarios, and analyze state-of-the-art deep neural network models for semantic segmentation. The dataset consists of around 2000 high-resolution aerial images, with annotated ground-truth data for semantic segmentation. We discuss the challenges of the dataset and train the state-of-the-art methods on this dataset to evaluate how well these methods can recognize the disaster situations. Finally, we discuss challenges for future research.
DEPFusion: Dual-Domain Enhancement and Priority-Guided Mamba Fusion for UAV Multispectral Object Detection
School of Information Engineering, Guangdong University of Technology（广东工业大学信息工程学院）；School of Electronic and Information Engineering, South China University of Technology（华南理工大学电子信息学院）；School of Artificial Intelligence, XiDian University（西安电子科技大学人工智能学院）
University of Southern California（南加州大学）；Institute for Creative Technologies（创意技术研究所）；US Army（美国陆军）
With state-of-the-art sensing and photogrammetric techniques, Microsoft Bing Maps team has created over 125 highly detailed 3D cities from 11 different countries that cover hundreds of thousands of square kilometer areas. The 3D city models were created using the photogrammetric technique with high-resolution images that were captured from aircraft-mounted cameras. Such a large 3D city database has caught the attention of the US Army for creating virtual simulation environments to support military operations. However, the 3D city models do not have semantic information such as buildings, veget…
School of Robotics, Hunan University（湖南大学机器人学院）；College of Electrical and Information Engineering, Hunan University（湖南大学电气与信息工程学院）；Technical University Munich（慕尼黑工业大学）
Although much significant progress has been made in the research field of object detection with deep learning, there still exists a challenging task for the objects with small size, which is notably pronounced in UAV-captured images. Addressing these issues, it is a critical need to explore the feature extraction methods that can extract more sufficient feature information of small objects. In this paper, we propose a novel method called Dense Multiscale Feature Fusion Pyramid Networks(DMFFPN), which is aimed at obtaining rich features as much as possible, improving the information propagation…
More Clear, More Flexible, More Precise: A Comprehensive Oriented Object Detection benchmark for UAV
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China（多媒体可信感知与高效计算重点实验室，中华人民共和国教育部）；Institute of Artificial Intelligence, Xiamen University（人工智能学院，厦门大学）；School of Informatics, Xiamen University（信息学院，厦门大学）；Fujian Engineering Research Center of Trusted Artificial Intelligence Analysis and Application, Institute of Artificial Intelligence, Xiamen University（福建工程研究中心可信人工智能分析与应用，人工智能学院，厦门大学）；Peng Cheng Laboratory, Shenzhen（鹏城实验室，深圳）
Unmanned aerial vehicles (UAVs) equipped with multiple complementary sensors have tremendous potential for fast autonomous or remote-controlled semantic scene analysis, e.g., for disaster examination. In this work, we propose a UAV system for real-time semantic inference and fusion of multiple sensor modalities. Semantic segmentation of LiDAR scans and RGB images, as well as object detection on RGB and thermal images, run online onboard the UAV computer using lightweight CNN architectures and embedded inference accelerators. We follow a late fusion approach where semantic information from mult…
EFSI-DETR: Efficient Frequency-Semantic Integration for Real-Time Small Object Detection in UAV Imagery
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing（信息工程测绘遥感国家重点实验室）；Wuhan University（武汉大学）；Wuhan University Shenzhen Research Institute（武汉大学深圳研究院）；School of Computer Science（计算机学院）；School of Automation Science and Engineering（自动化科学与工程学院）；South China University of Technology（华南理工大学）
Zero-Parameter Geometric Gating for Temporally Stable Low-Altitude UAV Video Semantic Segmentation
Beihang University（北京航空航天大学）；Northeastern University（东北大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Beijing Institute of Technology（北京理工大学）
Unmanned aerial vehicles (UAVs) equipped with multiple complementary sensors have tremendous potential for fast autonomous or remote-controlled semantic scene analysis, e.g., for disaster examination. Here, we propose a UAV system for real-time semantic inference and fusion of multiple sensor modalities. Semantic segmentation of LiDAR scans and RGB images, as well as object detection on RGB and thermal images, run online onboard the UAV computer using lightweight CNN architectures and embedded inference accelerators. We follow a late fusion approach where semantic information from multiple sen…
An object detection pipeline comprises a camera that captures the scene and an object detector that processes these images. The quality of the images directly affects the performance of the object detector. Many works nowadays focus either on improving the image quality or improving the object detection models independently, but neglect the importance of joint optimization of the two subsystems. The goal of this paper is to tune the detection throughput and accuracy of existing object detectors in the remote sensing scenario by focusing on optimizing the input images tailored to the object det…
UAVD-Mamba: Deformable Token Fusion Vision Mamba for Multimodal UAV Detection
College of Mechanical and Vehicle Engineering, Hunan University（湖南大学机械与车辆工程学院）；School of Electronic Information and Communications, Huazhong University of Science and Technology（华中科技大学电子信息与通信学院）
UAV-CB: A Complex-Background RGB-T Dataset and Local Frequency Bridge Network for UAV Detection
Pengcheng Laboratory（鹏城实验室）；South China University of Technology（南方科技大学）；Peking University（北京大学）；Xinjiang University（新疆大学）；Harbin Institute of Technology（哈尔滨工业大学）
Is your VLM Sky-Ready? A Comprehensive Spatial Intelligence Benchmark for UAV Navigation
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Peng Cheng Laboratory（鹏城实验室）；Xiaomi EV（小米电动汽车）；Georgia Institute of Technology（佐治亚理工学院）；Institute of Automation, CAS（中国科学院自动化研究所）；National University of Singapore（新加坡国立大学）
Unmanned aerial vehicle (UAV)-based visual object tracking has enabled a wide range of applications and attracted increasing attention in the field of intelligent transportation systems because of its versatility and effectiveness. As an emerging force in the revolutionary trend of deep learning, Siamese networks shine in UAV-based object tracking with their promising balance of accuracy, robustness, and speed. Thanks to the development of embedded processors and the gradual optimization of deep neural networks, Siamese trackers receive extensive research and realize preliminary combinations w…
RefGlass-GS: A UAV-Enabled Fusion Framework for Photorealistic, Semantic and Interactive Digitization of Reflective Glass Facades via Gaussian Splatting
Department of Civil and Environmental Engineering, The Hong Kong University of Science and Technology（香港科技大学土木与环境工程系）；Department of Civil and Environmental Engineering, University of Maryland（马里兰大学土木与环境工程系）；Department of the Built Environment, National University of Singapore（新加坡国立大学环境与建筑系）；Department of Architecture and Civil Engineering, City University of Hong Kong（香港城市大学建筑与土木工程系）
PRI-Net: A Lightweight Multimodal Framework for 3D UAV Localization
College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息与电子工程学院）；Zhejiang Key Laboratory of Multimodal Communication Networks and Intelligent Information Processing（浙江省多模态通信网络与智能信息处理重点实验室）
ReLATE: Reliability-Guided Evidence Fusion for Robust UAV--Satellite cross-view Geo-Localization
School of Electronics and Information Engineering, Harbin Institute of Technology（哈尔滨工业大学电子与信息工程学院）；National Key Laboratory of Radar Detection and Sensing, Nanjing Research Institute of Electronics Technology（南京电子技术研究所雷达探测与感知国家重点实验室）；School of Electrical and Electronic Engineering, University of Manchester（曼彻斯特大学电气与电子工程学院）
Tsinghua University（清华大学）；Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）
Selecting safe landing sites in non-cooperative environments is a key step towards the full autonomy of UAVs. However, the existing methods have the common problems of poor generalization ability and robustness. Their performance in unknown environments is significantly degraded and the error cannot be self-detected and corrected. In this paper, we construct a UAV system equipped with low-cost LiDAR and binocular cameras to realize autonomous landing in non-cooperative environments by detecting the flat and safe ground area. Taking advantage of the non-repetitive scanning and high FOV coverage…
Is Intermediate Fusion All You Need for UAV-based Collaborative Perception?
School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所认知与决策智能复杂系统重点实验室）
Vision-Based Anti Unmanned Aerial Technology: Opportunities and Challenges
College of Computer Science and Engineering, Guilin University of Technology, China（桂林理工大学计算机科学与工程学院）；Guangxi Key Laboratory of Embedded Technology and Intelligent System, Guilin University of Technology, China（广西嵌入式技术与智能系统重点实验室）；JD Logistic, Beijing, China（京东物流）
King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）
Autonomous UAV racing has recently emerged as an interesting research problem. The dream is to beat humans in this new fast-paced sport. A common approach is to learn an end-to-end policy that directly predicts controls from raw images by imitating an expert. However, such a policy is limited by the expert it imitates and scaling to other environments and vehicle dynamics is difficult. One approach to overcome the drawbacks of an end-to-end policy is to train a network only on the perception task and handle control with a PID or MPC controller. However, a single controller must be extensively …
UAV-Rain1k: A Benchmark for Raindrop Removal from UAV Aerial Imagery
Shenyang Aerospace University（沈阳航空航天大学）；Naval Aviation University（海军航空大学）；Nanjing University of Science and Technology（南京理工大学）；Zhejiang Ocean University（浙江海洋大学）
