This paper explores the process of designing an automatic multi-sensor drone detection system. Besides the common video and audio sensors, the system also includes a thermal infrared camera, which is shown to be a feasible solution to the drone detection task. Even with slightly lower resolution, the performance is just as good as a camera in visible range. The detector performance as a function of the sensor-to-target distance is also investigated. In addition, using sensor fusion, the system is made more robust than the individual sensors, helping to reduce false detections. To counteract the lack of public datasets, a novel video dataset containing 650 annotated infrared and visible videos of drones, birds, airplanes and helicopters is also presented (https://github.com/DroneDetectionThesis/Drone-detection-dataset). The database is complemented with an audio dataset of the classes drones, helicopters and background noise.
This paper reports a visible and thermal drone monitoring system that integrates deep-learning-based detection and tracking modules. The biggest challenge in adopting deep learning methods for drone detection is the paucity of training drone images especially thermal drone images. To address this issue, we develop two data augmentation techniques. One is a model-based drone augmentation technique that automatically generates visible drone images with a bounding box label on the drone's location. The other is exploiting an adversarial data augmentation methodology to create thermal drone images…
SAGA: Semantic-Aware Gray color Augmentation for Visible-to-Thermal Domain Adaptation across Multi-View Drone and Ground-Based Vision Systems
Department of Aerospace Engineering, Indian Institute of Science（航空航天工程系，印度科学研究院）；Robert Bosch Centre for Cyber Physical Systems, Indian Institute of Science（网络物理系统研究中心，印度科学研究院）
This paper presents a novel system for autonomous, vision-based drone racing combining learned data abstraction, nonlinear filtering, and time-optimal trajectory planning. The system has successfully been deployed at the first autonomous drone racing world championship: the 2019 AlphaPilot Challenge. Contrary to traditional drone racing systems, which only detect the next gate, our approach makes use of any visible gate and takes advantage of multiple, simultaneous gate detections to compensate for drift in the state estimate and build a global map of the gates. The global map and drift-compen…
Air Defence Regiment, Swedish Armed Forces（瑞典武装部队防空团）；Center for Applied Intelligent Systems Research (CAISR), Halmstad University（哈尔姆斯塔德大学应用智能系统研究中心）；RISE（瑞典研究院）
The use of small and remotely controlled unmanned aerial vehicles (UAVs), or drones, has increased in recent years. This goes in parallel with misuse episodes, with an evident threat to the safety of people or facilities. As a result, the detection of UAV has also emerged as a research topic. Most studies on drone detection fail to specify the type of acquisition device, the drone type, the detection range, or the dataset. The lack of proper UAV detection studies employing thermal infrared cameras is also an issue, despite its success with other targets. Besides, we have not found any previous…
The increase in the number of unmanned aerial vehicles a.k.a. drones pose several threats to public privacy, critical infrastructure and cyber security. Hence, detecting unauthorized drones is a significant problem which received attention in the last few years. In this paper, we present our experimental work on three drone detection methods (i.e., acoustic detection, radio frequency (RF) detection, and visual detection) to evaluate their efficacy in both indoor and outdoor environments. Owing to the limitations of these schemes, we present a novel encryption-based drone detection scheme that …
Object detection using aerial drone imagery has received a great deal of attention in recent years. While visible light images are adequate for detecting objects in most scenarios, thermal cameras can extend the capabilities of object detection to night-time or occluded objects. As such, RGB and Infrared (IR) fusion methods for object detection are useful and important. One of the biggest challenges in applying deep learning methods to RGB/IR object detection is the lack of available training data for drone IR imagery, especially at night. In this paper, we develop several strategies for creat…
Small Target Detection Based on Mask-Enhanced Attention Fusion of Visible and Infrared Remote Sensing Images
National Space Science Center, Chinese Academy of Sciences（中国科学院国家空间科学中心）；School of Computer Science and Technology, University of Chinese Academy of Sciences（中国科学院大学计算机科学与技术学院）；School of Electronic Engineering and Computer Science, Queen Mary University of London（伦敦大学玛丽女王学院电子工程与计算机科学学院）；School of Astronomy and Space Science, University of Chinese Academy of Sciences（中国科学院大学天文学与空间科学学院）
Vrije Universiteit Brussel（布鲁塞尔自由大学）；Université Libre de Bruxelles（布鲁塞尔自由大学）；VUB（布鲁塞尔自由大学）
The computer vision community has paid much attention to the development of visible image super-resolution (SR) using deep neural networks (DNNs) and has achieved impressive results. The advancement of non-visible light sensors, such as acoustic imaging sensors, has attracted much attention, as they allow people to visualize the intensity of sound waves beyond the visible spectrum. However, because of the limitations imposed on acquiring acoustic data, new methods for improving the resolution of the acoustic images are necessary. At this time, there is no acoustic imaging dataset designed for …
MoCTEFuse: Illumination-Gated Mixture of Chiral Transformer Experts for Multi-Level Infrared and Visible Image Fusion
School of Optics and Photonics, Beijing Institute of Technology（光学与光电学院，北京理工大学）；School of Computer Science and Technology, Beijing Institute of Technology（计算机科学与技术学院，北京理工大学）
Aligned Consensus Teaching for Label-Efficient Oriented Object Detection in Weakly-Aligned Visible-Infrared Imagery
Beijing University of Technology（北京工业大学）；University of Science and Technology of China（中国科学技术大学）；Peking University（北京大学）；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；Beijing Institute of Technology（北京理工大学）；Fuzhou University（福州大学）；China University of Geosciences Wuhan（中国地质大学（武汉））；Ghent University（根特大学）
Bridging Severe Cross-Modal Misalignment: End-to-End Visible-Infrared Object Detection via Explicit Feature-Domain Affine Registration
Beijing University of Technology（北京工业大学）；Central South University（中南大学）；Beijing Electronics Science & Technology Institute（北京电子科学技术研究所）；China Aerospace Science & Industry Corporation（中国航天科技集团）；Beijing Institute of Technology（北京理工大学）；Information Support Force Engineering University（信息支援部队工程大学）；Trunk Technology (Beijing) Co., Ltd.（trunk技术（北京）有限公司）
DPDETR: Decoupled Position Detection Transformer for Infrared-Visible Object Detection
School of Communications and Information Engineering, Chongqing University of Posts and Telecommunications（通信与信息工程学院，重庆邮电大学）；School of Intelligent Systems Engineering, Shenzhen Campus of Sun Yat-sen University（智能系统工程学院，中山大学深圳校区）；School of Mathematics and Statistics, Xi’an Jiaotong University（数学与统计学院，西安交通大学）
Advances in machine learning and deep neural networks for object detection, coupled with lower cost and power requirements of cameras, led to promising vision-based solutions for sUAS detection. However, solely relying on the visible spectrum has previously led to reliability issues in low contrast scenarios such as sUAS flying below the treeline and against bright sources of light. Alternatively, due to the relatively high heat signatures emitted from sUAS during flight, a long-wave infrared (LWIR) sensor is able to produce images that clearly contrast the sUAS from its background. However, c…
An FPGA smart camera implementation of segmentation models for drone wildfire imagery
Tecnologico de Monterrey（蒙特雷科技大学）；Hasso-Plattner Institute, University of Postdam（波茨坦大学哈索·普拉特纳研究所）；Universidad Autonoma de Guadalajara（瓜达拉哈拉自治大学）；Universidad de las Americas Puebla（普埃布拉美洲大学）；Università di Corsica（科西嘉大学）
University of Texas at Austin（德克萨斯大学奥斯汀分校）；Meta AI；Stanford University（斯坦福大学）；Reality Labs at Meta（Meta现实实验室）
We introduce the visual acoustic matching task, in which an audio clip is transformed to sound like it was recorded in a target environment. Given an image of the target environment and a waveform for the source audio, the goal is to re-synthesize the audio to match the target room acoustics as suggested by its visible geometry and materials. To address this novel task, we propose a cross-modal transformer model that uses audio-visual attention to inject visual properties into the audio and generate realistic audio output. In addition, we devise a self-supervised training objective that can le…
AlignFreeNet: Is Cross-Modal Pre-Alignment Necessary? An End-to-End Alignment-Free Lightweight Network for Visible-Infrared Object Detection
School of Computer Science, Jiangsu University of Technology（江苏科技大学计算机科学学院）；School of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics（南京航空航天大学计算机科学与技术学院）；School of Science and Technology, Hong Kong Metropolitan University（香港都会大学科技学院）；School of Data Science, Lingnan University（岭南大学数据科学学院）；Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）
P2Fusion: Prompt-based Progressive Infrared-Visible Image Fusion via Dual-Prior Distillation
Northwestern Polytechnical University（西北工业大学）；Hefei University of Technology（合肥工业大学）；Rocket Force University of Engineering（火箭军工程大学）；Chongqing University of Posts and Telecommunications（重庆邮电大学）
Integrating multispectral data in object detection, especially visible and infrared images, has received great attention in recent years. Since visible (RGB) and infrared (IR) images can provide complementary information to handle light variations, the paired images are used in many fields, such as multispectral pedestrian detection, RGB-IR crowd counting and RGB-IR salient object detection. Compared with natural RGB-IR images, we find detection in aerial RGB-IR images suffers from cross-modal weakly misalignment problems, which are manifested in the position, size and angle deviations of the …
FoCal: Frequency-Oriented Cross-Modal Interaction and Spectral Calibration for Aerial Visible-Infrared Object Detection
Nanjing University of Science and Technology（南京理工大学）；Shanghai Institute of Technical Physics, Chinese Academy of Sciences（中国科学院上海技术物理研究所）；China Electronics Technology Group Corporation（中国电子科技集团公司）
A UAV-Based VNIR Hyperspectral Benchmark Dataset for Landmine and UXO Detection
Rochester Institute of Technology, Chester F. Carlson Center for Imaging Science（罗切斯特理工学院，切斯特·F·卡尔森成像科学中心）；Lamont Doherty Earth Observatory, Columbia University（拉蒙特-多赫蒂地球观测站，哥伦比亚大学）；Demining Research Community（排雷研究社区）
Technical University of Cluj-Napoca（克卢日-纳波卡技术大学）；National Center for Atmospheric Research（美国国家大气研究中心）；Astronomical Institute of the Romanian Academy（罗马尼亚科学院天文研究所）
Sunquakes are seismic emissions visible on the solar surface, associated with some solar flares. Although discovered in 1998, they have only recently become a more commonly detected phenomenon. Despite the availability of several manual detection guidelines, to our knowledge, the astrophysical data produced for sunquakes is new to the field of Machine Learning. Detecting sunquakes is a daunting task for human operators and this work aims to ease and, if possible, to improve their detection. Thus, we introduce a dataset constructed from acoustic egression-power maps of solar active regions obta…
COXNet: Cross-Layer Fusion with Adaptive Alignment and Scale Integration for RGBT Tiny Object Detection
Beijing Institute of Technology（北京理工大学）；National Astronomical Observatories, Chinese Academy of Sciences（中国科学院国家天文台）；National Key Laboratory of Space Target Awareness, Space Engineering University（航天工程大学空间目标感知全国重点实验室）；Key Laboratory of Photoelectronic Imaging Technology and System, Ministry of Education of China（教育部光电成像技术与系统重点实验室）；Chongqing Innovation Center, Beijing Institute of Technology（北京理工大学重庆创新中心）；China North Vehicle Research Institute（中国北方车辆研究所）
When One Modality Is Not Enough: Multimodal Sex and Life-Stage Classification of Red Deer from Aerial RGB-Thermal Video
Aalborg University（奥尔堡大学）；University of Applied Sciences Upper Austria（上奥地利应用科学大学）；University of Novi Sad（诺威萨德大学）；Massachusetts Institute of Technology（麻省理工学院）
