Nighttime UAV tracking under low-illuminated scenarios has achieved great progress by domain adaptation (DA). However, previous DA training-based works are deficient in narrowing the discrepancy of temporal contexts for UAV trackers. To address the issue, this work proposes a prompt-driven temporal domain adaptation training framework to fully utilize temporal contexts for challenging nighttime UAV tracking, i.e., TDA. Specifically, the proposed framework aligns the distribution of temporal contexts from daytime and nighttime domains by training the temporal feature generator against the discriminator. The temporal-consistent discriminator progressively extracts shared domain-specific features to generate coherent domain discrimination results in the time series. Additionally, to obtain high-quality training samples, a prompt-driven object miner is employed to precisely locate objects in unannotated nighttime videos. Moreover, a new benchmark for long-term nighttime UAV tracking is constructed. Exhaustive evaluations on both public and self-constructed nighttime benchmarks demonstrate the remarkable performance of the tracker trained in TDA framework, i.e., TDA-Track. Real-world tests at nighttime also show its practicality. The code and demo videos are available at https://github.com/vision4robotics/TDA-Track.
Previous advances in object tracking mostly reported on favorable illumination circumstances while neglecting performance at nighttime, which significantly impeded the development of related aerial robot applications. This work instead develops a novel unsupervised domain adaptation framework for nighttime aerial tracking (named UDAT). Specifically, a unique object discovery approach is provided to generate training patches from raw nighttime tracking videos. To tackle the domain discrepancy, we employ a Transformer-based bridging layer post to the feature extractor to align image features fro…
Guilin University of Technology（桂林理工大学）；Southern University of Science and Technology（南方科技大学）；Sichuan University（四川大学）
Unmanned aerial vehicle (UAV) tracking has wide potential applications in such as agriculture, navigation, and public security. However, the limitations of computing resources, battery capacity, and maximum load of UAV hinder the deployment of deep learning-based tracking algorithms on UAV. Consequently, discriminative correlation filters (DCF) trackers stand out in the UAV tracking community because of their high efficiency. However, their precision is usually much lower than trackers based on deep learning. Model compression is a promising way to narrow the gap (i.e., effciency, precision) b…
Event-RGB Adaptive Tracking for Nighttime Highway Perception
School of Intelligent Systems Engineering, Sun Yat-sen University（中山大学智能系统工程学院）；School of Intelligence Science and Engineering, Harbin Institute of Technology（哈尔滨工业大学智能科学与工程学院）
Unmanned aerial vehicles (UAV) have been widely used in various fields, and their invasion of security and privacy has aroused social concern. Several detection and tracking systems for UAVs have been introduced in recent years, but most of them are based on radio frequency, radar, and other media. We assume that the field of computer vision is mature enough to detect and track invading UAVs. Thus we propose a visible light mode dataset called Dalian University of Technology Anti-UAV dataset, DUT Anti-UAV for short. It contains a detection dataset with a total of 10,000 images and a tracking d…
University of Chinese Academic of Sciences（中国科学院大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Institute of North Electronic Equipment（北方电子设备研究所）；Baidu Research（百度研究院）
Unmanned Aerial Vehicle (UAV) offers lots of applications in both commerce and recreation. With this, monitoring the operation status of UAVs is crucially important. In this work, we consider the task of tracking UAVs, providing rich information such as location and trajectory. To facilitate research on this topic, we propose a dataset, Anti-UAV, with more than 300 video pairs containing over 580k manually annotated bounding boxes. The releasing of such a large-scale dataset could be a useful initial step in research of tracking UAVs. Furthermore, the advancement of addressing research challen…
Shanghai Jiao Tong University（上海交通大学）；CloudWalk Technology Co., Ltd（云从科技有限公司）；Shenzhen Research Institute of Big Data, the Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）深圳大数据研究院）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；JD Explore Academy in JD.com（京东集团京东探索研究院）；University of Sydney（悉尼大学）
Unmanned aerial vehicle (UAV) tracking is of great significance for a wide range of applications, such as delivery and agriculture. Previous benchmarks in this area mainly focused on small-scale tracking problems while ignoring the amounts of data, types of data modalities, diversities of target categories and scenarios, and evaluation protocols involved, greatly hiding the massive power of deep UAV tracking. In this work, we propose WebUAV-3M, the largest public UAV tracking benchmark to date, to facilitate both the development and evaluation of deep UAV trackers. WebUAV-3M contains over 3.3 …
How Far are Modern Trackers from UAV-Anti-UAV? A Million-Scale Benchmark and New Baseline
Shanghai Jiao Tong University（上海交通大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；CloudWalk Technology Co., Ltd（云walk科技有限公司）；School of Artificial Intelligence, Shanghai Jiao Tong University（上海交通大学人工智能学院）；School of Aeronautics and Astronautics, Sun Yat-sen University（中山大学航空宇航学院）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）
Layer-Guided UAV Tracking: Enhancing Efficiency and Occlusion Robustness
Department of Control Science and Engineering, University of Shanghai for Science and Technology（上海理工大学控制科学与工程学院）；Busness School, University of Shanghai for Science and Technology（上海理工大学商学院）
Evidential Detection and Tracking Collaboration: New Problem, Benchmark and Algorithm for Robust Anti-UAV System
Jiangnan University（江南大学）；Institute of North Electronic Equipment（北方电子设备研究所）；Intelligent Game and Decision Laboratory（智能博弈与决策实验室）；Peng Cheng Laboratory（鹏城实验室）；National University of Singapore（新加坡国立大学）；Beijing Institute of Basic Medical Sciences（北京基础医学研究所）；Beijing Institute of Technology（北京理工大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Beijing University of Posts and Telecommunications（北京邮电大学）；PhiGent Robotics（鉴智机器人）；Tsinghua University（清华大学）
Infrared UAV Target Tracking with Dynamic Feature Refinement and Global Contextual Attention Knowledge Distillation
School of Computer Science and Technology, Xidian University（西安电子科技大学计算机科学与技术学院）；Xi’an Modern Control Technology Research Institute（西安现代控制技术研究所）；School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）
A Tri-Modal Dataset and a Baseline System for Tracking Unmanned Aerial Vehicles
School of Artificial Intelligence and Computer Science, Jiangnan University（江南大学人工智能与计算机科学学院）；School of Computer Science and Electronic Engineering and the Centre for Vision, Speech and Signal Processing, University of Surrey（萨里大学计算机科学与电子工程学院及视觉、语音与信号处理中心）
Singapore University of Technology and Design（新加坡科技设计大学）；Shandong University（山东大学）
Human behavior understanding with unmanned aerial vehicles (UAVs) is of great significance for a wide range of applications, which simultaneously brings an urgent demand of large, challenging, and comprehensive benchmarks for the development and evaluation of UAV-based models. However, existing benchmarks have limitations in terms of the amount of captured data, types of data modalities, categories of provided tasks, and diversities of subjects and environments. Here we propose a new benchmark - UAVHuman - for human behavior understanding with UAVs, which contains 67,428 multi-modal video sequ…
Tongji University（同济大学）；National University of Singapore（新加坡国立大学）；The University of Hong Kong（香港大学）
The outstanding computational efficiency of discriminative correlation filter (DCF) fades away with various complicated improvements. Previous appearances are also gradually forgotten due to the exponential decay of historical views in traditional appearance updating scheme of DCF framework, reducing the model's robustness. In this work, a novel tracker based on DCF framework is proposed to augment memory of previously appeared views while running at real-time speed. Several historical views and the current view are simultaneously introduced in training to allow the tracker to adapt to new app…
Learning Occlusion-Robust Vision Transformers for Real-Time UAV Tracking
College of Computer Science and Engineering, Guilin University of Technology（桂林理工大学计算机科学与工程学院）；School of Computer Science, Fudan University（复旦大学计算机科学技术学院）；School of Artificial Intelligence, Sun Yat-sen University（中山大学人工智能学院）
Dalian University of Technology（大连理工大学）；Peng Cheng Laboratory（鹏城实验室）；Tiwaki Co.Ltd.（Tiwaki有限公司）
With the popularity of multi-modal sensors, visible-thermal (RGB-T) object tracking is to achieve robust performance and wider application scenarios with the guidance of objects' temperature information. However, the lack of paired training samples is the main bottleneck for unlocking the power of RGB-T tracking. Since it is laborious to collect high-quality RGB-T sequences, recent benchmarks only provide test sequences. In this paper, we construct a large-scale benchmark with high diversity for visible-thermal UAV tracking (VTUAV), including 500 sequences with 1.7 million high-resolution (192…
Prior correlation filter (CF)-based tracking methods for unmanned aerial vehicles (UAVs) have virtually focused on tracking in the daytime. However, when the night falls, the trackers will encounter more harsh scenes, which can easily lead to tracking failure. In this regard, this work proposes a novel tracker with anti-dark function (ADTrack). The proposed method integrates an efficient and effective low-light image enhancer into a CF-based tracker. Besides, a target-aware mask is simultaneously generated by virtue of image illumination variation. The target-aware mask can be applied to joint…
Unmanned Aerial Vehicles (UAV) can pose a major risk for aviation safety, due to both negligent and malicious use. For this reason, the automated detection and tracking of UAV is a fundamental task in aerial security systems. Common technologies for UAV detection include visible-band and thermal infrared imaging, radio frequency and radar. Recent advances in deep neural networks (DNNs) for image-based object detection open the possibility to use visual information for this detection and tracking task. Furthermore, these detection architectures can be implemented as backbones for visual trackin…
MambaNUT: Nighttime UAV Tracking via Mamba-based Adaptive Curriculum Learning
College of Computer Science and Engineering, Guilin University of Technology（桂林理工大学计算机科学与工程学院）；School of Computer Science, Fudan University（复旦大学计算机科学学院）；School of Artificial Intelligence, Sun Yat-sen University（中山大学人工智能学院）
Multi-Modal UAV Detection, Classification and Tracking Algorithm -- Technical Report for CVPR 2024 UG2 Challenge
Shanghai Jiao Tong University（上海交通大学）；JITRI（江苏集萃智能制造技术研究所有限公司）；Dalian University of Technology（大连理工大学）；University of Electronic Science and Technology of China（电子科技大学）；The University of Tokyo（东京大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
UAV-Track VLA: Embodied Aerial Tracking via Vision-Language-Action Models
Beijing Institute of Technology（北京理工大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Sanya（三亚学院）；Beijing University of Posts and Telecommunications（北京邮电大学）；Hunan University（湖南大学）；Beihang University（北京航空航天大学）；Flying Intelligence Team (Virtual Research Community)（飞行智能团队（虚拟研究社区））
Dual Prompt-Driven Feature Encoding for Nighttime UAV Tracking
Pratt School of Engineering, Duke University（杜克大学工程学院）；School of Mechanical Engineering, Tongji University（同济大学机械工程学院）；Shanghai Key Laboratory of Wearable Robotics and Human Machine Interaction, Tongji University（同济大学可穿戴机器人与人机交互重点实验室）；School of Computing and Data Science, The University of Hong Kong（香港大学计算与数据科学学院）
Most previous progress in object tracking is realized in daytime scenes with favorable illumination. State-of-the-arts can hardly carry on their superiority at night so far, thereby considerably blocking the broadening of visual tracking-related unmanned aerial vehicle (UAV) applications. To realize reliable UAV tracking at night, a spatial-channel Transformer-based low-light enhancer (namely SCT), which is trained in a novel task-inspired manner, is proposed and plugged prior to tracking approaches. To achieve semantic-level low-light enhancement targeting the high-level task, the novel spati…
Beijing Institute of Technology（北京理工大学）；Key Laboratory of Digital Earth Science, Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院数字地球科学重点实验室）；Mississippi State University（密西西比州立大学）
Owing to effective and flexible data acquisition, unmanned aerial vehicle (UAV) has recently become a hotspot across the fields of computer vision (CV) and remote sensing (RS). Inspired by recent success of deep learning (DL), many advanced object detection and tracking approaches have been widely applied to various UAV-related tasks, such as environmental monitoring, precision agriculture, traffic management. This paper provides a comprehensive survey on the research progress and prospects of DL-based UAV object detection and tracking methods. More specifically, we first outline the challenge…
UAUTrack: Towards Unified Multimodal Anti-UAV Visual Tracking
College of Intelligent Robotics and Advanced Manufacturing（智能机器人与先进制造学院）；Fudan University（复旦大学）；School of Computer Science and Technology（计算机科学与技术学院）；Zhejiang Normal University（浙江师范大学）；Department of Mechanical Engineering（机械工程系）；The Hong Kong Polytechnic University（香港理工大学）
Unmanned aerial vehicle (UAV)-based tracking is attracting increasing attention and developing rapidly in applications such as agriculture, aviation, navigation, transportation and public security. Recently, discriminative correlation filters (DCF)-based trackers have stood out in UAV tracking community for their high efficiency and appealing robustness on a single CPU. However, due to limited onboard computation resources and other challenges the efficiency and accuracy of existing DCF-based approaches is still not satisfying. In this paper, we explore using segmentation by the GrabCut to imp…
Unmanned aerial vehicle (UAV)-based visual object tracking has enabled a wide range of applications and attracted increasing attention in the field of intelligent transportation systems because of its versatility and effectiveness. As an emerging force in the revolutionary trend of deep learning, Siamese networks shine in UAV-based object tracking with their promising balance of accuracy, robustness, and speed. Thanks to the development of embedded processors and the gradual optimization of deep neural networks, Siamese trackers receive extensive research and realize preliminary combinations w…
Beijing Institute of Technology（北京理工大学）；Space Engineering University（航天工程大学）；Chongqing Innovation Center, Beijing Institute of Technology（北京理工大学重庆创新中心）
Learning Motion Blur Robust Vision Transformers for Real-Time UAV Tracking
College of Computer Science（计算机科学学院）；Guilin University of Technology（桂林理工大学）；School of Computer Science（计算机科学学院）；Fudan University（复旦大学）；Sun Yat-sen University（中山大学）；Sichuan University（四川大学）
STATrack: A Target-Aware Fully Spiking Neural Network for Efficient UAV Tracking
College of Computer Science and Engineering, Guilin University of Technology（桂林理工大学计算机科学与工程学院）；School of Artificial Intelligence, Sun Yat-sen University（中山大学人工智能学院）；School of Electronic Information, Central South University（中南大学电子信息学院）
TrackingMiM: Efficient Mamba-in-Mamba Serialization for Real-time UAV Object Tracking
Department of Electronic and Electrical Engineering, Southern University of Science and Technology（电子与电气工程系，南方科技大学）；Peng Cheng Laboratory（鹏城实验室）；Department of Applied Mathematics and Theoretical Physics, University of Cambridge（应用数学与理论物理系，剑桥大学）；State Grid Corporation of China（国家电网公司）；East China Institute of Computing Technology（东部计算技术研究所）；Shenzhen Key Laboratory of Robotics and Computer Vision（深圳机器人与计算机视觉重点实验室）
Dynamic Semantic-Aware Correlation Modeling for UAV Tracking
College of Computer Science and Artificial Intelligence, Fudan University（计算机科学与人工智能学院，复旦大学）；College of Intelligent Robotics and Advanced Manufacturing, Fudan University（智能机器人与先进制造学院，复旦大学）
UAV (Unmanned Aerial Vehicles): Diverse Applications of UAV Datasets in Segmentation, Classification, Detection, and Tracking
Silicon Orchard Research and Analytics Lab（Silicon Orchard 研究与分析实验室）；Daffodil International University（水仙花国际大学）；BRAC University（BRAC 大学）；Texas Tech University（德克萨斯理工大学）；Clark Atlanta University（克拉克亚特兰大大学）
Tongji University（同济大学）；National University of Singapore（新加坡国立大学）；Technical University of Munich（慕尼黑工业大学）；The University of Hong Kong（香港大学）
Correlation filter-based tracking has been widely applied in unmanned aerial vehicle (UAV) with high efficiency. However, it has two imperfections, i.e., boundary effect and filter corruption. Several methods enlarging the search area can mitigate boundary effect, yet introducing undesired background distraction. Existing frame-by-frame context learning strategies for repressing background distraction nevertheless lower the tracking speed. Inspired by keyframe-based simultaneous localization and mapping, keyfilter is proposed in visual tracking for the first time, in order to handle the above …
MambaTrack: Exploiting Dual-Enhancement for Night UAV Tracking
Cooperative Medianet Innovation Center（合作 medi 创新中心）；Shanghai Jiao Tong University（上海交通大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；CloudWalk Technology（云从科技）；Shanghai AI Laboratory（上海人工智能实验室）
CGTrack: Cascade Gating Network with Hierarchical Feature Aggregation for UAV Tracking
Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences（杭州高等研究院，中国科学院大学）；Dept. of Computer Science and Engineering, University of North Texas（计算机科学与工程系，北卡罗来纳州立大学）；Institute of Software Chinese Academy of Science（中国科学院软件研究所）
Recently, the Siamese-based method has stood out from multitudinous tracking methods owing to its state-of-the-art (SOTA) performance. Nevertheless, due to various special challenges in UAV tracking, \textit{e.g.}, severe occlusion and fast motion, most existing Siamese-based trackers hardly combine superior performance with high efficiency. To this concern, in this paper, a novel attentional Siamese tracker (SiamAPN++) is proposed for real-time UAV tracking. By virtue of the attention mechanism, we conduct a special attentional aggregation network (AAN) consisting of self-AAN and cross-AAN fo…
