UTPTrack: Towards Simple and Unified Token Pruning for Visual Tracking
Institute of Digital Twin, Eastern Institute of Technology（数字孪生研究所，东部技术研究所）；Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative（宁波空间智能与数字衍生关键实验室）；Shanghai Jiao Tong University（上海交通大学）；The Hong Kong Polytechnic University（香港理工大学）；Munich Center for Machine Learning, LMU Munich（慕尼黑机器学习中心，慕尼黑大学）
One-stream Transformer-based trackers achieve advanced performance in visual object tracking but suffer from significant computational overhead that hinders real-time deployment. While token pruning offers a path to efficiency, existing methods are fragmented. They typically prune the search region, dynamic template, and static template in isolation, overlooking critical inter-component dependencies, which yields suboptimal pruning and degraded accuracy. To address this, we introduce UTPTrack, a simple and Unified Token Pruning framework that, for the first time, jointly compresses all three components. UTPTrack employs an attention-guided, token type-aware strategy to holistically model redundancy, a design that seamlessly supports unified tracking across multimodal and language-guided tasks within a single model. Extensive evaluations on 10 benchmarks demonstrate that UTPTrack achieves a new state-of-the-art in the accuracy-efficiency trade-off for pruning-based trackers, pruning 65.4% of vision tokens in RGB-based tracking and 67.5% in unified tracking while preserving 99.7% and 100.5% of baseline performance, respectively. This strong performance across both RGB and multimodal scenarios underlines its potential as a robust foundation for future research in efficient visual tracking. Code will be released at https://github.com/EIT-NLP/UTPTrack.
We propose a novel meta-learning framework for real-time object tracking with efficient model adaptation and channel pruning. Given an object tracker, our framework learns to fine-tune its model parameters in only a few iterations of gradient-descent during tracking while pruning its network channels using the target ground-truth at the first frame. Such a learning problem is formulated as a meta-learning task, where a meta-tracker is trained by updating its meta-parameters for initial weights, learning rates, and pruning masks through carefully designed tracking simulations. The integrated me…
Object trackers deployed on low-power devices need to be light-weight, however, most of the current state-of-the-art (SOTA) methods rely on using compute-heavy backbones built using CNNs or transformers. Large sizes of such models do not allow their deployment in low-power conditions and designing compressed variants of large tracking models is of great importance. This paper demonstrates how highly compressed light-weight object trackers can be designed using neural architectural pruning of large CNN and transformer based trackers. Further, a comparative study on architectural choices best su…
Existing Visual Object Tracking (VOT) only takes the target area in the first frame as a template. This causes tracking to inevitably fail in fast-changing and crowded scenes, as it cannot account for changes in object appearance between frames. To this end, we revamped the tracking framework with Progressive Context Encoding Transformer Tracker (ProContEXT), which coherently exploits spatial and temporal contexts to predict object motion trajectories. Specifically, ProContEXT leverages a context-aware self-attention module to encode the spatial and temporal context, refining and updating the …
Guilin University of Technology（桂林理工大学）；Southern University of Science and Technology（南方科技大学）；Sichuan University（四川大学）
Unmanned aerial vehicle (UAV) tracking has wide potential applications in such as agriculture, navigation, and public security. However, the limitations of computing resources, battery capacity, and maximum load of UAV hinder the deployment of deep learning-based tracking algorithms on UAV. Consequently, discriminative correlation filters (DCF) trackers stand out in the UAV tracking community because of their high efficiency. However, their precision is usually much lower than trackers based on deep learning. Model compression is a promising way to narrow the gap (i.e., effciency, precision) b…
In this paper the research on optimisation of visual object tracking using a Siamese neural network for embedded vision systems is presented. It was assumed that the solution shall operate in real-time, preferably for a high resolution video stream, with the lowest possible energy consumption. To meet these requirements, techniques such as the reduction of computational precision and pruning were considered. Brevitas, a tool dedicated for optimisation and quantisation of neural networks for FPGA implementation, was used. A number of training scenarios were tested with varying levels of optimis…
Adaptively Bypassing Vision Transformer Blocks for Efficient Visual Tracking
College of Computer Science and Engineering, Guilin University of Technology（桂林理工大学计算机科学与工程学院）；Department of Computer Science and Engineering, Southern University of Science and Technology（南方科技大学计算机科学与工程系）；College of Computer Science, Sichuan University（四川大学计算机学院）
How to perform effective information fusion of different modalities is a core factor in boosting the performance of RGBT tracking. This paper presents a novel deep fusion algorithm based on the representations from an end-to-end trained convolutional neural network. To deploy the complementarity of features of all layers, we propose a recursive strategy to densely aggregate these features that yield robust representations of target objects in each modality. In different modalities, we propose to prune the densely aggregated features of all modalities in a collaborative way. In a specific, we e…
TSMS-SAM2: Multi-scale Temporal Sampling Augmentation and Memory-Splitting Pruning for Promptable Video Object Segmentation and Tracking in Surgical Scenarios
University of Texas Southwestern Medical Center（德克萨斯大学西南医学中心）
OptiPMB: Enhancing 3D Multi-Object Tracking with Optimized Poisson Multi-Bernoulli Filtering
School of Electronic Information Engineering, Beihang University（电子信息工程学院，北京航空航天大学）；School of Automation and Intelligent Sensing, Shanghai Jiaotong University（自动化与智能感知学院，上海交通大学）；Department of Computer Science and Engineering, The Hong Kong University of Science and Technology（计算机科学与工程系，香港科技大学）；College of Science and Engineering, James Cook University（科学与工程学院，詹姆斯库克大学）；School of Artificial Intelligence and Computer Science, Nantong University（人工智能与计算机科学学院，南通大学）；Faculty of Data Science, City University of Macau（数据科学学院，澳门城市大学）；Research Laboratory of Smart Driving and Intelligent Transportation Systems, Southeast University（智能驾驶与智能交通系统研究实验室，东南大学）
Context-Aware Token Pruning and Discriminative Selective Attention for Transformer Tracking
Department of Computational Mathematics, University of Moratuwa（计算数学系，摩图瓦大学）；Department of Interdisciplinary Studies, University of Jaffna（跨学科研究系，贾夫纳大学）；Department of Computer Science, University of Jaffna（计算机科学系，贾夫纳大学）
Towards Compact Unified Multimodal Tracking: Synergizing Knowledge Distillation with Structural Pruning
The City College of New York（纽约城市学院）；INSAIT；Sofia University “St. Kliment Ohridski”（圣克莱门特奥赫里德斯基索非亚大学）；Julius-Maximilians-Universität Würzburg（维尔茨堡大学）；State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所人工智能安全国家重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Shenzhen University of Advanced Technology（深圳先进技术研究院）
Pruning enables appealing reductions in network memory footprint and time complexity. Conventional post-training pruning techniques lean towards efficient inference while overlooking the heavy computation for training. Recent exploration of pre-training pruning at initialization hints on training cost reduction via pruning, but suffers noticeable performance degradation. We attempt to combine the benefits of both directions and propose a policy that prunes as early as possible during training without hurting performance. Instead of pruning at initialization, our method exploits initial dense t…
Hanyang University（汉阳大学）；Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）
In this paper, we propose a new joint object detection and tracking (JoDT) framework for 3D object detection and tracking based on camera and LiDAR sensors. The proposed method, referred to as 3D DetecTrack, enables the detector and tracker to cooperate to generate a spatio-temporal representation of the camera and LiDAR data, with which 3D object detection and tracking are then performed. The detector constructs the spatio-temporal features via the weighted temporal aggregation of the spatial features obtained by the camera and LiDAR fusion. Then, the detector reconfigures the initial detecti…
University of Electronic Science and Technology of China（电子科技大学）；Shenzhen Loop Area Institute（深圳河套学院）；Northumbria University（北umbria大学）；The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen)（香港中文大学（深圳））
GARAD-SLAM: 3D GAussian splatting for Real-time Anti Dynamic SLAM
School of Information and Communication Engineering, Dalian University of Technology（大连理工大学信息与通信工程学院）；School of Aeronautics and Astronautics, Sun Yat-sen University（中山大学航空航天学院）；Faculty of Science and Technology, University of Macau（澳门大学科技学院）
PiVoT: A Variational Solution for Real-time Large-scale Multi-object Detection and Tracking under Heavy Clutter
Institute for Imaging, Data and Communications (IDCOM), University of Edinburgh（爱丁堡大学成像、数据与通信研究所（IDCOM））；Department of Engineering, University of Cambridge（剑桥大学工程系）；School of Mathematics, University of Edinburgh（爱丁堡大学数学学院）
