arXiv 2503.114962025-05-27cs.CV

面向指代性多目标跟踪的认知解耦

Cognitive Disentanglement for Referring Multi-Object Tracking

Qingdao Institute of Software, College of Computer Science and Technology, China University of Petroleum (East China)（青岛软件研究所，计算机科学与技术学院，中国石油大学（华东））；Thrust of AI, Hong Kong University of Science and Technology (GuangZhou)（人工智能研究组，香港科学与技术大学（广州））；Shandong Key Laboratory of Intelligent Oil & Gas Industrial Software（山东省智能油气工业软件重点实验室）；Wangxuan Institute of Computer Technology, Peking University（王轩计算机技术研究所，北京大学）；School of Automation, Southeast University（自动化学院，东南大学）；School of Computer Science, Beijing Institute of Technology（计算机科学学院，北京理工大学）；School of Artificial Intelligence and Computer Science, Nantong University（人工智能与计算机科学学院，南通大学）；Thrust of Intelligent Transportation, Hong Kong University of Science and Technology (GuangZhou)（智能交通研究组，香港科学与技术大学（广州））；Department of Electrical Engineering and Electronics, University of Liverpool（电子工程与电子学院，利物浦大学）

Shaofeng Liang, Runwei Guan, Wangwang Lian, Daizong Liu, Xiaolou Sun, Dongming Wu, Yutao Yue, Weiping Ding, Hui Xiong

备注：: 27 pages, 12 figures

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对指代性多目标跟踪中语言特征与视觉特征融合不充分的问题，提出CDRMT框架，借鉴人类视觉“what”和“where”通路实现语言解耦与分层注入，并在重构语言表示的约束下显著提升了跟踪性能。

摘要：

作为多源信息融合在智能交通感知系统中的重要应用，指代性多目标跟踪（RMOT）涉及基于语言指代在视频序列中定位和跟踪特定目标。然而，现有的RMOT方法通常将语言描述视为整体嵌入，难以将语言表达中包含的丰富语义信息与视觉特征有效整合。在需要综合理解静态目标属性和空间运动信息的复杂场景中，这一局限性尤为明显。在本文中，我们提出了一种面向指代性多目标跟踪的认知解耦（CDRMT）框架来解决这些挑战。它将人类视觉处理系统中的“what”和“where”通路适配到RMOT任务中。具体而言，我们的框架首先建立跨模态连接，同时保留模态特定的特征。然后，它解耦语言描述并将其分层注入到目标查询中，从粗粒度到细粒度的语义层面细化目标理解。最后，我们基于视觉特征重构语言表示，确保被跟踪的目标忠实地反映指代表达。在不同基准数据集上的大量实验表明，CDRMT相比最先进的方法取得了显著改进，在Refer-KITTI上的HOTA得分平均提升6.0%，在Refer-KITTI-V2上平均提升3.2%。我们的方法推进了RMOT领域的最先进水平，同时为多源信息融合提供了新见解。

英文摘要：

As a significant application of multi-source information fusion in intelligent transportation perception systems, Referring Multi-Object Tracking (RMOT) involves localizing and tracking specific objects in video sequences based on language references. However, existing RMOT approaches often treat language descriptions as holistic embeddings and struggle to effectively integrate the rich semantic information contained in language expressions with visual features. This limitation is especially apparent in complex scenes requiring comprehensive understanding of both static object attributes and spatial motion information. In this paper, we propose a Cognitive Disentanglement for Referring Multi-Object Tracking (CDRMT) framework that addresses these challenges. It adapts the "what" and "where" pathways from the human visual processing system to RMOT tasks. Specifically, our framework first establishes cross-modal connections while preserving modality-specific characteristics. It then disentangles language descriptions and hierarchically injects them into object queries, refining object understanding from coarse to fine-grained semantic levels. Finally, we reconstruct language representations based on visual features, ensuring that tracked objects faithfully reflect the referring expression. Extensive experiments on different benchmark datasets demonstrate that CDRMT achieves substantial improvements over state-of-the-art methods, with average gains of 6.0% in HOTA score on Refer-KITTI and 3.2% on Refer-KITTI-V2. Our approach advances the state-of-the-art in RMOT while simultaneously providing new insights into multi-source information fusion.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。