Single visual object tracking from an unmanned aerial vehicle (UAV) poses fundamental challenges such as object occlusion, small-scale objects, background clutter, and abrupt camera motion. To tackle these difficulties, we propose to integrate the 3D structure of the observed scene into a detection-by-tracking algorithm. We introduce a pipeline that combines a model-free visual object tracker, a sparse 3D reconstruction, and a state estimator. The 3D reconstruction of the scene is computed with an image-based Structure-from-Motion (SfM) component that enables us to leverage a state estimator in the corresponding 3D scene during tracking. By representing the position of the target in 3D space rather than in image space, we stabilize the tracking during ego-motion and improve the handling of occlusions, background clutter, and small-scale objects. We evaluated our approach on prototypical image sequences, captured from a UAV with low-altitude oblique views. For this purpose, we adapted an existing dataset for visual object tracking and reconstructed the observed scene in 3D. The experimental results demonstrate that the proposed approach outperforms methods using plain visual cues as well as approaches leveraging image-space-based state estimations. We believe that our approach can be beneficial for traffic monitoring, video surveillance, and navigation.
Unmanned aerial vehicle (UAV)-based visual object tracking has enabled a wide range of applications and attracted increasing attention in the field of intelligent transportation systems because of its versatility and effectiveness. As an emerging force in the revolutionary trend of deep learning, Siamese networks shine in UAV-based object tracking with their promising balance of accuracy, robustness, and speed. Thanks to the development of embedded processors and the gradual optimization of deep neural networks, Siamese trackers receive extensive research and realize preliminary combinations w…
UAUTrack: Towards Unified Multimodal Anti-UAV Visual Tracking
College of Intelligent Robotics and Advanced Manufacturing（智能机器人与先进制造学院）；Fudan University（复旦大学）；School of Computer Science and Technology（计算机科学与技术学院）；Zhejiang Normal University（浙江师范大学）；Department of Mechanical Engineering（机械工程系）；The Hong Kong Polytechnic University（香港理工大学）
Learning Motion Blur Robust Vision Transformers for Real-Time UAV Tracking
College of Computer Science（计算机科学学院）；Guilin University of Technology（桂林理工大学）；School of Computer Science（计算机科学学院）；Fudan University（复旦大学）；Sun Yat-sen University（中山大学）；Sichuan University（四川大学）
STATrack: A Target-Aware Fully Spiking Neural Network for Efficient UAV Tracking
College of Computer Science and Engineering, Guilin University of Technology（桂林理工大学计算机科学与工程学院）；School of Artificial Intelligence, Sun Yat-sen University（中山大学人工智能学院）；School of Electronic Information, Central South University（中南大学电子信息学院）
TrackingMiM: Efficient Mamba-in-Mamba Serialization for Real-time UAV Object Tracking
Department of Electronic and Electrical Engineering, Southern University of Science and Technology（电子与电气工程系，南方科技大学）；Peng Cheng Laboratory（鹏城实验室）；Department of Applied Mathematics and Theoretical Physics, University of Cambridge（应用数学与理论物理系，剑桥大学）；State Grid Corporation of China（国家电网公司）；East China Institute of Computing Technology（东部计算技术研究所）；Shenzhen Key Laboratory of Robotics and Computer Vision（深圳机器人与计算机视觉重点实验室）
Tongji University（同济大学）；National University of Singapore（新加坡国立大学）；Technical University of Munich（慕尼黑工业大学）；The University of Hong Kong（香港大学）
Correlation filter-based tracking has been widely applied in unmanned aerial vehicle (UAV) with high efficiency. However, it has two imperfections, i.e., boundary effect and filter corruption. Several methods enlarging the search area can mitigate boundary effect, yet introducing undesired background distraction. Existing frame-by-frame context learning strategies for repressing background distraction nevertheless lower the tracking speed. Inspired by keyframe-based simultaneous localization and mapping, keyfilter is proposed in visual tracking for the first time, in order to handle the above …
MambaTrack: Exploiting Dual-Enhancement for Night UAV Tracking
Cooperative Medianet Innovation Center（合作 medi 创新中心）；Shanghai Jiao Tong University（上海交通大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；CloudWalk Technology（云从科技）；Shanghai AI Laboratory（上海人工智能实验室）
CGTrack: Cascade Gating Network with Hierarchical Feature Aggregation for UAV Tracking
Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences（杭州高等研究院，中国科学院大学）；Dept. of Computer Science and Engineering, University of North Texas（计算机科学与工程系，北卡罗来纳州立大学）；Institute of Software Chinese Academy of Science（中国科学院软件研究所）
Tongji University（同济大学）；Hong Kong Polytechnic University（香港理工大学）
Correlation filter (CF)-based methods have demonstrated exceptional performance in visual object tracking for unmanned aerial vehicle (UAV) applications, but suffer from the undesirable boundary effect. To solve this issue, spatially regularized correlation filters (SRDCF) proposes the spatial regularization to penalize filter coefficients, thereby significantly improving the tracking performance. However, the temporal information hidden in the response maps is not considered in SRDCF, which limits the discriminative power and the robustness for accurate tracking. This work proposes a novel ap…
Tongji University（同济大学）；Hong Kong Polytechnic University（香港理工大学）
Correlation filter (CF) has recently exhibited promising performance in visual object tracking for unmanned aerial vehicle (UAV). Such online learning method heavily depends on the quality of the training-set, yet complicated aerial scenarios like occlusion or out of view can reduce its reliability. In this work, a novel time slot-based distillation approach is proposed to efficiently and effectively optimize the training-set's quality on the fly. A cooperative energy minimization function is established to score the historical samples adaptively. To accelerate the scoring process, frames with…
Visual object tracking, which is representing a major interest in image processing field, has facilitated numerous real world applications. Among them, equipping unmanned aerial vehicle (UAV) with real time robust visual trackers for all day aerial maneuver, is currently attracting incremental attention and has remarkably broadened the scope of applications of object tracking. However, prior tracking methods have merely focused on robust tracking in the well-illuminated scenes, while ignoring trackers' capabilities to be deployed in the dark. In darkness, the conditions can be more complex and…
Current unmanned aerial vehicle (UAV) visual tracking algorithms are primarily limited with respect to: (i) the kind of size variation they can deal with, (ii) the implementation speed which hardly meets the real-time requirement. In this work, a real-time UAV tracking algorithm with powerful size estimation ability is proposed. Specifically, the overall tracking task is allocated to two 2D filters: (i) translation filter for location prediction in the space domain, (ii) size filter for scale and aspect ratio optimization in the size domain. Besides, an efficient two-stage re-detection strateg…
An Event-based Fast Intensity Reconstruction Scheme for UAV Real-time Perception
Hangzhou International Innovation Institute, Beihang University（杭州国际创新院，北京航空航天大学）；School of Aeronautic Science and Engineering, Beihang University（航空科学与工程学院，北京航空航天大学）；Institute of Unmanned System, Beihang University（无人系统研究院，北京航空航天大学）；Tianmushan Laboratory, Xixi Octagon, Yuhang, Hangzhou（天幕实验室，西溪圆，余杭，杭州）
Tongji University（同济大学）；the University of Hong Kong（香港大学）
Vision-based object tracking has boosted extensive autonomous applications for unmanned aerial vehicles (UAVs). However, the dynamic changes in flight maneuver and viewpoint encountered in UAV tracking pose significant difficulties, e.g. , aspect ratio change, and scale variation. The conventional cross-correlation operation, while commonly used, has limitations in effectively capturing perceptual similarity and incorporates extraneous background information. To mitigate these limitations, this work presents a novel saliency-guided dynamic vision Transformer (SGDViT) for UAV tracking. The prop…
Aerial tracking, which has exhibited its omnipresent dedication and splendid performance, is one of the most active applications in the remote sensing field. Especially, unmanned aerial vehicle (UAV)-based remote sensing system, equipped with a visual tracking approach, has been widely used in aviation, navigation, agriculture,transportation, and public security, etc. As is mentioned above, the UAV-based aerial tracking platform has been gradually developed from research to practical application stage, reaching one of the main aerial remote sensing technologies in the future. However, due to t…
Visual tracking has yielded promising applications with unmanned aerial vehicle (UAV). In literature, the advanced discriminative correlation filter (DCF) type trackers generally distinguish the foreground from the background with a learned regressor which regresses the implicit circulated samples into a fixed target label. However, the predefined and unchanged regression target results in low robustness and adaptivity to uncertain aerial tracking scenarios. In this work, we exploit the local maximum points of the response map generated in the detection phase to automatically locate current di…
Unmanned aerial vehicle (UAV) based visual tracking has been confronted with numerous challenges, e.g., object motion and occlusion. These challenges generally introduce unexpected mutations of target appearance and result in tracking failure. However, prevalent discriminative correlation filter (DCF) based trackers are insensitive to target mutations due to a predefined label, which concentrates on merely the centre of the training region. Meanwhile, appearance mutations caused by occlusion or similar objects usually lead to the inevitable learning of wrong information. To cope with appearanc…
Jet Propulsion Laboratory, California Institute of Technology（加州理工学院喷气推进实验室）
We propose TRADE for robust tracking and 3D localization of a moving target in cluttered environments, from UAVs equipped with a single camera. Ultimately TRADE enables 3d-aware target following. Tracking-by-detection approaches are vulnerable to target switching, especially between similar objects. Thus, TRADE predicts and incorporates the target 3D trajectory to select the right target from the tracker's response map. Unlike static environments, depth estimation of a moving target from a single camera is a ill-posed problem. Therefore we propose a novel 3D localization method for ground targ…
M$^2$E-UAV: A Benchmark and Analysis for Onboard Motion-on-Motion Event-Based Tiny UAV Detection
Fujian Key Laboratory of Urban Intelligent Sensing and Computing, School of Informatics, Xiamen University, Xiamen, China（福建城市智能感知与计算重点实验室，厦门大学信息学院，厦门，中国）；Meta, Menlo Park, USA（Meta，Menlo Park，美国）
Guilin University of Technology（桂林理工大学）；Southern University of Science and Technology（南方科技大学）；Sichuan University（四川大学）
Unmanned aerial vehicle (UAV) tracking has wide potential applications in such as agriculture, navigation, and public security. However, the limitations of computing resources, battery capacity, and maximum load of UAV hinder the deployment of deep learning-based tracking algorithms on UAV. Consequently, discriminative correlation filters (DCF) trackers stand out in the UAV tracking community because of their high efficiency. However, their precision is usually much lower than trackers based on deep learning. Model compression is a promising way to narrow the gap (i.e., effciency, precision) b…
Unmanned aerial vehicles (UAV) have been widely used in various fields, and their invasion of security and privacy has aroused social concern. Several detection and tracking systems for UAVs have been introduced in recent years, but most of them are based on radio frequency, radar, and other media. We assume that the field of computer vision is mature enough to detect and track invading UAVs. Thus we propose a visible light mode dataset called Dalian University of Technology Anti-UAV dataset, DUT Anti-UAV for short. It contains a detection dataset with a total of 10,000 images and a tracking d…
University of Chinese Academic of Sciences（中国科学院大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Institute of North Electronic Equipment（北方电子设备研究所）；Baidu Research（百度研究院）
Unmanned Aerial Vehicle (UAV) offers lots of applications in both commerce and recreation. With this, monitoring the operation status of UAVs is crucially important. In this work, we consider the task of tracking UAVs, providing rich information such as location and trajectory. To facilitate research on this topic, we propose a dataset, Anti-UAV, with more than 300 video pairs containing over 580k manually annotated bounding boxes. The releasing of such a large-scale dataset could be a useful initial step in research of tracking UAVs. Furthermore, the advancement of addressing research challen…
This paper describes a computationally-enhanced M100 UAV platform with an onboard deep learning inference system for integrated computer vision and navigation able to autonomously find and visually identify by coat pattern individual Holstein Friesian cattle in freely moving herds. We propose an approach that utilises three deep convolutional neural network architectures running live onboard the aircraft; that is, a YoloV2-based species detector, a dual-stream CNN delivering exploratory agency and an InceptionV3-based biometric LRCN for individual animal identification. We evaluate the perform…
Engineering Research Institute（工程研究所）；High Perf. Comp. Research Lab.（高性能计算实验室）；Auckland University of Technology（奥克兰理工大学）
In this work we consider the task of detecting sheep onboard an unmanned aerial vehicle (UAV) flying at an altitude of 80 m. At this height, the sheep are relatively small, only about 15 pixels across. Although deep learning strategies have gained enormous popularity in the last decade and are now extensively used for object detection in many fields, state-of-the-art detectors perform poorly in the case of smaller objects. We develop a novel dataset of UAV imagery of sheep and consider a variety of object detectors to determine which is the most suitable for our task in terms of both accuracy …
Beijing Institute of Technology（北京理工大学）；Key Laboratory of Digital Earth Science, Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院数字地球科学重点实验室）；Mississippi State University（密西西比州立大学）
Owing to effective and flexible data acquisition, unmanned aerial vehicle (UAV) has recently become a hotspot across the fields of computer vision (CV) and remote sensing (RS). Inspired by recent success of deep learning (DL), many advanced object detection and tracking approaches have been widely applied to various UAV-related tasks, such as environmental monitoring, precision agriculture, traffic management. This paper provides a comprehensive survey on the research progress and prospects of DL-based UAV object detection and tracking methods. More specifically, we first outline the challenge…
Shanghai Jiao Tong University（上海交通大学）；CloudWalk Technology Co., Ltd（云从科技有限公司）；Shenzhen Research Institute of Big Data, the Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）深圳大数据研究院）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；JD Explore Academy in JD.com（京东集团京东探索研究院）；University of Sydney（悉尼大学）
Unmanned aerial vehicle (UAV) tracking is of great significance for a wide range of applications, such as delivery and agriculture. Previous benchmarks in this area mainly focused on small-scale tracking problems while ignoring the amounts of data, types of data modalities, diversities of target categories and scenarios, and evaluation protocols involved, greatly hiding the massive power of deep UAV tracking. In this work, we propose WebUAV-3M, the largest public UAV tracking benchmark to date, to facilitate both the development and evaluation of deep UAV trackers. WebUAV-3M contains over 3.3 …
Unmanned aerial vehicle (UAV)-based tracking is attracting increasing attention and developing rapidly in applications such as agriculture, aviation, navigation, transportation and public security. Recently, discriminative correlation filters (DCF)-based trackers have stood out in UAV tracking community for their high efficiency and appealing robustness on a single CPU. However, due to limited onboard computation resources and other challenges the efficiency and accuracy of existing DCF-based approaches is still not satisfying. In this paper, we explore using segmentation by the GrabCut to imp…
Kyoto University（京都大学）；Japan Science and Technology Agency（日本科学技术振兴机构）
Automatic classification of trees using remotely sensed data has been a dream of many scientists and land use managers. Recently, Unmanned aerial vehicles (UAV) has been expected to be an easy-to-use, cost-effective tool for remote sensing of forests, and deep learning has attracted attention for its ability concerning machine vision. In this study, using a commercially available UAV and a publicly available package for deep learning, we constructed a machine vision system for the automatic classification of trees. In our method, we segmented a UAV photography image of forest into individual t…
Beijing Institute of Technology（北京理工大学）；Space Engineering University（航天工程大学）；Chongqing Innovation Center, Beijing Institute of Technology（北京理工大学重庆创新中心）
How Far are Modern Trackers from UAV-Anti-UAV? A Million-Scale Benchmark and New Baseline
Shanghai Jiao Tong University（上海交通大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；CloudWalk Technology Co., Ltd（云walk科技有限公司）；School of Artificial Intelligence, Shanghai Jiao Tong University（上海交通大学人工智能学院）；School of Aeronautics and Astronautics, Sun Yat-sen University（中山大学航空宇航学院）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）
Dynamic Semantic-Aware Correlation Modeling for UAV Tracking
College of Computer Science and Artificial Intelligence, Fudan University（计算机科学与人工智能学院，复旦大学）；College of Intelligent Robotics and Advanced Manufacturing, Fudan University（智能机器人与先进制造学院，复旦大学）
Layer-Guided UAV Tracking: Enhancing Efficiency and Occlusion Robustness
Department of Control Science and Engineering, University of Shanghai for Science and Technology（上海理工大学控制科学与工程学院）；Busness School, University of Shanghai for Science and Technology（上海理工大学商学院）
UAV (Unmanned Aerial Vehicles): Diverse Applications of UAV Datasets in Segmentation, Classification, Detection, and Tracking
Silicon Orchard Research and Analytics Lab（Silicon Orchard 研究与分析实验室）；Daffodil International University（水仙花国际大学）；BRAC University（BRAC 大学）；Texas Tech University（德克萨斯理工大学）；Clark Atlanta University（克拉克亚特兰大大学）
Evidential Detection and Tracking Collaboration: New Problem, Benchmark and Algorithm for Robust Anti-UAV System
Jiangnan University（江南大学）；Institute of North Electronic Equipment（北方电子设备研究所）；Intelligent Game and Decision Laboratory（智能博弈与决策实验室）；Peng Cheng Laboratory（鹏城实验室）；National University of Singapore（新加坡国立大学）；Beijing Institute of Basic Medical Sciences（北京基础医学研究所）；Beijing Institute of Technology（北京理工大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Beijing University of Posts and Telecommunications（北京邮电大学）；PhiGent Robotics（鉴智机器人）；Tsinghua University（清华大学）
Infrared UAV Target Tracking with Dynamic Feature Refinement and Global Contextual Attention Knowledge Distillation
School of Computer Science and Technology, Xidian University（西安电子科技大学计算机科学与技术学院）；Xi’an Modern Control Technology Research Institute（西安现代控制技术研究所）；School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院）
A Tri-Modal Dataset and a Baseline System for Tracking Unmanned Aerial Vehicles
School of Artificial Intelligence and Computer Science, Jiangnan University（江南大学人工智能与计算机科学学院）；School of Computer Science and Electronic Engineering and the Centre for Vision, Speech and Signal Processing, University of Surrey（萨里大学计算机科学与电子工程学院及视觉、语音与信号处理中心）
