Context-Aware Integration of Language and Visual References for Natural Language Tracking
Zhejiang University of Technology(浙江工业大学);Nanyang Technological University(南洋理工大学);Zhejiang University(浙江大学);The Hong Kong University of Science and Technology(香港科技大学)
Tracking by natural language specification (TNL) aims to consistently localize a target in a video sequence given a linguistic description in the initial frame. Existing methodologies perform language-based and template-based matching for target reasoning separately and merge the matching results from two sources, which suffer from tracking drift when language and visual templates miss-align with the dynamic target state and ambiguity in the later merging stage. To tackle the issues, we propose a joint multi-modal tracking framework with 1) a prompt modulation module to leverage the complementarity between temporal visual templates and language expressions, enabling precise and context-aware appearance and linguistic cues, and 2) a unified target decoding module to integrate the multi-modal reference cues and executes the integrated queries on the search image to predict the target location in an end-to-end manner directly. This design ensures spatio-temporal consistency by leveraging historical visual information and introduces an integrated solution, generating predictions in a single step. Extensive experiments conducted on TNL2K, OTB-Lang, LaSOT, and RefCOCOg validate the efficacy of our proposed approach. The results demonstrate competitive performance against state-of-the-art methods for both tracking and grounding.
Sparse Message Passing Network with Feature Integration for Online Multiple Object Tracking
Nanjing University of Science and Technology(南京理工大学);Graz University of Technology(格拉茨技术大学)
现有的多目标跟踪(MOT)方法设计了复杂的体系结构以获得更好的跟踪性能。然而,没有一个适当的组织输入信息,他们仍然无法执行跟踪鲁棒性和遭受频繁的身份切换。在本文中,我们提出了两个新的方法,连同一个简单的在线消息传递网络(MPN),以解决这些限制。首先,我们探索了图节点和边嵌入的不同集成方法,并提出了一种新的IoU(Intersection over Union)引导函数,该函数改善了长期跟踪并处理身份切换。其次,我们引入了一种分层采样策略来构建稀疏图,从而可以将训练集中在更困难的样本上。实验结果表明,一个简单的在线MPN与这两个贡献可以执行更好地比许多国家的最先进的方法。此外,我们的关联方法具有很好的推广性,也可以改善基于私有检测的方法的结果。
Model-free Tracking with Deep Appearance and Motion Features Integration
Beihang University(北京航空航天大学);Ministry of Industry and Information Technology of China(中华人民共和国工业和信息化部);Beijing Advanced Innovation Center for Big Data-Based Precision Medicine(北京大数据精准医疗高精尖创新中心)
Unifying Visual and Vision-Language Tracking via Contrastive Learning
Deep Space Exploration Laboratory(深空探测实验室);University of Science and Technology of China(中国科学技术大学);Intelligent Science Technology Academy of CASIC(中国航天科工集团智能科学技术研究院)
IMT Atlantique(IMT大西洋高等矿业电信学院);LaTIM UMR 1101, Inserm(LaTIM(INSERM UMR 1101));ICube UMR 7357, CNRS, Université de Strasbourg(斯特拉斯堡大学CNRS ICube(UMR 7357));Université de Bretagne Occidentale(布列塔尼西部大学)
CiteTracker: Correlating Image and Text for Visual Tracking
Peng Cheng Laboratory(鹏城实验室);Harbin Institute of Technology Shenzhen(哈尔滨工业大学(深圳));Dalian University of Technology(大连理工大学);UC Merced(加州大学默塞德分校);Yonsei University(延世大学)
Pierre and Marie Curie University(皮埃尔和玛丽居里大学);ESPCI Paris, PSL Research University(巴黎高等物理化工学院,巴黎文理研究大学);National University of Defense Technology(国防科技大学);Peking University(北京大学);Sun Yat-Sen University(中山大学)