SFTrack++: A Fast Learnable Spectral Segmentation Approach for Space-Time Consistent Tracking
Bitdefender（比特梵德）；University of Bucharest（布加勒斯特大学）；Institute of Mathematics of the Romanian Academy（罗马尼亚科学院数学研究所）
Elena Burceanu
备注：
Accepted at Neural Information Processing Systems (NeurIPS) 2020 - Pre-registration Workshop and at The International Conference on Computer Vision (ICCV) 2021 - Structured Representations for Video Understanding Workshop
We propose an object tracking method, SFTrack++, that smoothly learns to preserve the tracked object consistency over space and time dimensions by taking a spectral clustering approach over the graph of pixels from the video, using a fast 3D filtering formulation for finding the principal eigenvector of this graph's adjacency matrix. To better capture complex aspects of the tracked object, we enrich our formulation to multi-channel inputs, which permit different points of view for the same input. The channel inputs are in our experiments, the output of multiple tracking methods. After combining them, instead of relying only on hidden layers representations to predict a good tracking bounding box, we explicitly learn an intermediate, more refined one, namely the segmentation map of the tracked object. This prevents the rough common bounding box approach to introduce noise and distractors in the learning process. We test our method, SFTrack++, on five tracking benchmarks: OTB, UAV, NFS, GOT-10k, and TrackingNet, using five top trackers as input. Our experimental results validate the pre-registered hypothesis. We obtain consistent and robust results, competitive on the three traditional benchmarks (OTB, UAV, NFS) and significantly on top of others (by over $1.1\%$ on accuracy) on GOT-10k and TrackingNet, which are newer, larger, and more varied datasets.
Institute of Automation Chinese Academy of Sciences (CASIA)（中国科学院自动化研究所）；TuSimple（图森未来）；School of Artificial Intelligence University of Chinese Academy of Sciences (UCAS)（中国科学院大学人工智能学院）；Centre for Artificial Intelligence and Robotics HKISI_CAS（香港中科院人工智能与机器人中心）
Data association across frames is at the core of Multiple Object Tracking (MOT) task. This problem is usually solved by a traditional graph-based optimization or directly learned via deep learning. Despite their popularity, we find some points worth studying in current paradigm: 1) Existing methods mostly ignore the context information among tracklets and intra-frame detections, which makes the tracker hard to survive in challenging cases like severe occlusion. 2) The end-to-end association methods solely rely on the data fitting power of deep neural networks, while they hardly utilize the adv…
Towards Low-Latency Event Stream-based Visual Object Tracking: A Slow-Fast Approach
School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院）；Beijing Institute of Technology（北京理工大学）；School of Electronic and Information Engineering, Anhui University（安徽大学电子与信息工程学院）；Peng Cheng Laboratory（鹏城实验室）；School of Computer Science, Peking University（北京大学计算机科学学院）；School of Electronic and Computer Engineering, Shenzhen Graduate School, Peking University（北京大学深圳研究生院电子与计算机工程学院）
Tracking of objects in 3D is a fundamental task in computer vision that finds use in a wide range of applications such as autonomous driving, robotics or augmented reality. Most recent approaches for 3D multi object tracking (MOT) from LIDAR use object dynamics together with a set of handcrafted features to match detections of objects. However, manually designing such features and heuristics is cumbersome and often leads to suboptimal performance. In this work, we instead strive towards a unified and learning based approach to the 3D MOT problem. We design a graph structure to jointly process …
We propose a novel approach for joint 3D multi-object tracking and reconstruction from RGB-D sequences in indoor environments. To this end, we detect and reconstruct objects in each frame while predicting dense correspondences mappings into a normalized object space. We leverage those correspondences to inform a graph neural network to solve for the optimal, temporally-consistent 7-DoF pose trajectories of all objects. The novelty of our method is two-fold: first, we propose a new graph-based approach for differentiable pose estimation over time to learn optimal pose trajectories; second, we p…
MotionTrack: Learning Motion Predictor for Multiple Object Tracking
School of Computer Science, National University of Defense Technology（国防科技大学计算机学院）；JD Explore Academy（京东探索研究院）；Meituan Inc.（美团公司）；Laboratory of Digitizing Software for Frontier Equipment, National University of Defense Technology（国防科技大学前沿装备数字化软件实验室）
The recent trend in multiple object tracking (MOT) is heading towards leveraging deep learning to boost the tracking performance. However, it is not trivial to solve the data-association problem in an end-to-end fashion. In this paper, we propose a novel proposal-based learnable framework, which models MOT as a proposal generation, proposal scoring and trajectory inference paradigm on an affinity graph. This framework is similar to the two-stage object detector Faster RCNN, and can solve the MOT problem in a data-driven way. For proposal generation, we propose an iterative graph clustering met…
EPIPTrack: Rethinking Prompt Modeling with Explicit and Implicit Prompts for Multi-Object Tracking
College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University（吉林大学教育部长春符号计算与知识工程重点实验室）；Yangtze University（扬子大学）
Recent works have shown that convolutional networks have substantially improved the performance of multiple object tracking by simultaneously learning detection and appearance features. However, due to the local perception of the convolutional network structure itself, the long-range dependencies in both the spatial and temporal cannot be obtained efficiently. To incorporate the spatial layout, we propose to exploit the local correlation module to model the topological relationship between targets and their surrounding environment, which can enhance the discriminative power of our model in cro…
GateMOT: Q-Gated Attention for Dense Object Tracking
Huazhong University of Science and Technology, Wuhan, China（华中科技大学，武汉，中国）；Zhejiang University, Hangzhou, China（浙江大学，杭州，中国）；La Trobe University, Melbourne, Australia（拉筹伯大学，墨尔本，澳大利亚）
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Software Chinese Academy of Sciences（中国科学院软件研究所）；University at Albany, State University of New York（纽约州立大学奥尔巴尼分校）；JD Finance America Corporation（京东金融美国公司）
In this paper, we present a novel siamese motion-aware network (SiamMan) for visual tracking, which consists of the siamese feature extraction subnetwork, followed by the classification, regression, and localization branches in parallel. The classification branch is used to distinguish the foreground from background, and the regression branch is adopt to regress the bounding box of target. To reduce the impact of manually designed anchor boxes to adapt to different target motion patterns, we design the localization branch, which aims to coarsely localize the target to help the regression branc…
School of Computer Science and Technology, Anhui University（安徽大学计算机科学与技术学院）；School of Electronic and Information Engineering, Anhui University（安徽大学电子与信息工程学院）
Combining traditional RGB cameras with bio-inspired event cameras for robust object tracking has garnered increasing attention in recent years. However, most existing multimodal tracking algorithms depend heavily on high-complexity Vision Transformer architectures for feature extraction and fusion across modalities. This not only leads to substantial computational overhead but also limits the effectiveness of cross-modal interactions. In this paper, we propose an efficient RGB-Event object tracking framework based on the linear-complexity Vision Mamba network, termed Mamba-FETrack V2. Specific…
S.T.A.R.-Track: Latent Motion Models for End-to-End 3D Object Tracking with Adaptive Spatio-Temporal Appearance Representations
Mercedes-Benz AG（梅赛德斯-奔驰公司）；University of Tübingen（蒂宾根大学）；Institute for Intelligent Systems, Esslingen University of Applied Sciences（埃斯林根应用科学大学智能系统研究所）
Engineering an Efficient Object Tracker for Non-Linear Motion
Department of Computer Science, Faculty of Mathematics, University of Belgrade（计算机科学系，数学系，贝尔格莱德大学）；Department of Signals and Systems, School of Electrical Engineering, University of Belgrade（信号与系统系，电气工程学院，贝尔格莱德大学）；Center for Bussiness Decision Making, Faculty of Organizational sciences, University of Belgrade（商业决策中心，组织科学系，贝尔格莱德大学）
Institute for Artificial Intelligence at Tsinghua University（清华大学人工智能学院）；State Key Laboratory of Intelligent Technology and Systems（智能技术与系统国家重点实验室）；Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）；Center for Intelligent Connected Vehicles and Transportation（智能网联汽车与交通中心）；Department of Computer Sicence and Technology, Tsinghua University（清华大学计算机科学与技术系）
In a complex road traffic scene, illegal lane intrusion of pedestrians or cyclists constitutes one of the main safety challenges in autonomous driving application. In this paper, we propose a novel object-level phase space reconstruction network (PSRNet) for motion time series classification, aiming to recognize lane intrusion actions that occur 150m ahead through a monocular camera fixed on moving vehicle. In the PSRNet, the movement of pedestrians and cyclists, specifically viewed as an observable object-level dynamic process, can be reconstructed as trajectories of state vectors in a latent…
Xidian University（西安电子科技大学）；Temple University（天普大学）；University of Idaho（爱达荷大学）
The Localization of the target object for data retrieval is a key issue in the Intelligent and Connected Transportation Systems (ICTS). However, due to lack of intelligence in the traditional transportation system, it can take tremendous resources to manually retrieve and locate the queried objects among a large number of images. In order to solve this issue, we propose an effective method to query-based object localization that uses artificial intelligence techniques to automatically locate the queried object in the complex background. The presented method is termed as Fine-grained and Progre…
Gaze tracking is an important technology in many domains. Techniques such as Convolutional Neural Networks (CNN) has allowed the invention of gaze tracking method that relies only on commodity hardware such as the camera on a personal computer. It has been shown that the full-face region for gaze estimation can provide better performance than from an eye image alone. However, a problem with using the full-face image is the heavy computation due to the larger image size. This study tackles this problem through compression of the input full-face image by removing redundant information using a no…
Beijing Institute of Technology（北京理工大学）；Inception Institute of Artificial Intelligence（起源人工智能研究院）；Mohamed Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Recently, Siamese-based trackers have achieved promising performance in visual tracking. Most recent Siamese-based trackers typically employ a depth-wise cross-correlation (DW-XCorr) to obtain multi-channel correlation information from the two feature maps (target and search region). However, DW-XCorr has several limitations within Siamese-based tracking: it can easily be fooled by distractors, has fewer activated channels, and provides weak discrimination of object boundaries. Further, DW-XCorr is a handcrafted parameter-free module and cannot fully benefit from offline learning on large-scal…
Robust RGB-T Tracking via Learnable Visual Fourier Prompt Fine-tuning and Modality Fusion Prompt Generation
Key Laboratory of Education Blockchain and Intelligent Technology, Ministry of Education, Guangxi Normal University（教育区块链与智能技术重点实验室，教育部，广西师范大学）；Guangxi Key Lab of Multi-Source Information Mining and Security, Guangxi Normal University（多源信息挖掘与安全广西重点实验室，广西师范大学）
LGBP-OrgaNet: Learnable Gaussian Band Pass Fusion of CNN and Transformer Features for Robust Organoid Segmentation and Tracking
School of Optoelectronic Science and Engineering（光电科学与工程学院）；Translational Chinese Medicine Key Laboratory of Sichuan, Sichuan-Chongqing Joint Key Laboratory of innovation of New Drugs of Traditional, Chinese Medicine Sichuan Institute for Translational Chinese Medicine（四川 translational 中医关键实验室，四川-重庆联合新药创新中药关键实验室，四川 translational 中医研究所）；Sichuan Cancer Hospital and Institute（四川癌症医院与研究所）
Differentiable solvers for the linear assignment problem (LAP) have attracted much research attention in recent years, which are usually embedded into learning frameworks as components. However, previous algorithms, with or without learning strategies, usually suffer from the degradation of the optimality with the increment of the problem size. In this paper, we propose a learnable linear assignment solver based on deep graph networks. Specifically, we first transform the cost matrix to a bipartite graph and convert the assignment task to the problem of selecting reliable edges from the constr…
TrackAny3D: Transferring Pretrained 3D Models for Category-unified 3D Point Cloud Tracking
Zhejiang University of Technology（浙江工业大学）；RMIT University（皇家墨尔本理工大学）；Zhejiang Key Laboratory of Visual Information Intelligent Processing（浙江省视觉信息智能处理重点实验室）
State University of New York at Buffalo（纽约州立大学布法罗分校）；Amazon（亚马逊）
Video Instance Segmentation (VIS) aims to simultaneously classify, segment, and track multiple object instances in videos. Recent clip-level VIS takes a short video clip as input each time showing stronger performance than frame-level VIS (tracking-by-segmentation), as more temporal context from multiple frames is utilized. Yet, most clip-level methods are neither end-to-end learnable nor real-time. These limitations are addressed by the recent VIS transformer (VisTR) which performs VIS end-to-end within a clip. However, VisTR suffers from long training time due to its frame-wise dense attenti…
3D object detection from monocular image(s) is a challenging and long-standing problem of computer vision. To combine information from different perspectives without troublesome 2D instance tracking, recent methods tend to aggregate multiview feature by sampling regular 3D grid densely in space, which is inefficient. In this paper, we attempt to improve multi-view feature aggregation by proposing a learnable keypoints sampling method, which scatters pseudo surface points in 3D space, in order to keep data sparsity. The scattered points augmented by multi-view geometric constraints and visual f…
The Chinese University of Hong Kong（香港中文大学）；The University of Hong Kong（香港大学）；MEGVII Technology（旷视科技）；SmartMore（思谋科技）
In this work, we present a unified framework for multi-modality 3D object detection, named UVTR. The proposed method aims to unify multi-modality representations in the voxel space for accurate and robust single- or cross-modality 3D detection. To this end, the modality-specific space is first designed to represent different inputs in the voxel feature space. Different from previous work, our approach preserves the voxel space without height compression to alleviate semantic ambiguity and enable spatial connections. To make full use of the inputs from different sensors, the cross-modality inte…
The automated analysis of microscopy images is a challenge in the context of single-cell tracking and quantification. This work has as goals the study of the performance of deep learning for segmenting microscopy images and the improvement of the previously available pipeline for tracking single cells. Deep learning techniques, mainly convolutional neural networks, have been applied to cell segmentation problems and have shown high accuracy and fast performance. To perform the image segmentation, an analysis of hyperparameters was done in order to implement a convolutional neural network with …
