发表机构
School of Computer Science and Technology, Tianjin University; Faculty of Computer Science and Artificial Intelligence, Shenzhen University of Advanced Technology(天津大学计算机科学与技术学院; 深圳先进技术大学计算机科学与人工智能学院)
机构由 AI 辅助整理,请以论文原文为准。AI 中文总结
研究跨域引用多目标跟踪(CD-RMOT)问题,构建CD-RMOT-Bench基准并提供QCA框架,实验发现域转移严重影响RMOT性能,QCA建立基线,CD-RMOT-Bench为跨视觉域的鲁棒语言引导跟踪开辟新方向。
AI 中文摘要
引用多目标跟踪(RMOT)将跟踪从类别驱动的感知扩展到语言引导的理解,通过将物体轨迹与自然语言表达相结合。尽管最近有进展,但现有RMOT研究大多在域内设置下进行,未探索视觉域转移下语言条件跟踪的鲁棒性。本文研究跨域引用多目标跟踪(CD-RMOT),这是一个新的挑战性问题,评估在有标签源域训练的RMOT模型能否在不同视觉条件的无标签目标域中可靠地遵循自然语言表达。为支持系统研究,构建了CD-RMOT-Bench基准,结合真实清晰域引用跟踪数据、对齐的数字孪生变体和真实不利域视频。还提供了以查询为中心的适应(QCA)框架。实验表明域转移严重降低RMOT性能,QCA建立了强大基线,CD-RMOT-Bench为跨视觉域的鲁棒语言引导跟踪开辟了新方向。
英文摘要
Referring multi-object tracking (RMOT) extends tracking from category-driven perception to language-guided understanding by grounding object trajectories in natural-language expressions. Despite recent progress, existing RMOT studies are largely conducted under in-domain settings, leaving the robustness of language-conditioned tracking under inevitable visual domain shifts unexplored. In this paper, we study Cross-Domain Referring Multi-Object Tracking (CD-RMOT), a new and challenging problem that evaluates whether an RMOT model trained on a labeled source domain can reliably follow natural-language expressions in an unlabeled target domain with different visual conditions. To support systematic study, we construct CD-RMOT-Bench, a unified benchmark that combines real clear-domain referring tracking data, aligned digital-twin variants, and real adverse-domain videos. CD-RMOT-Bench enables both controlled weather/viewpoint shift analysis and realistic synthetic-real transfer evaluation under a shared RMOT protocol. Further, we provide a Query-Centric Adaptation (QCA) framework, designed to stabilize the query space that bridges visual trajectories and referring expressions. Extensive experiments reveal that domain shifts severely degrade RMOT performance, where the failure is not merely caused by object detection errors but more critically by unstable expression-conditioned temporal association and target selection. QCA establishes a strong baseline, while CD-RMOT-Bench opens a new direction for robust language-guided tracking across visual domains.