模板-搜索域自适应通过多阶段特征对齐用于跨模态目标跟踪

中文摘要 AI 辅助

视觉目标跟踪通常假设初始模板和后续搜索帧共享相同的传感模态。在实践中，传感器的可用性或操作可能随时间变化，在模板和搜索帧之间产生显著的表示差距。与成对模态同时可用的传统多模态跟踪不同，跨模态跟踪要求当模板和搜索帧来自不同的活动模态时进行定位。因此，我们引入了TSDA-Track，一个模板-搜索域自适应框架，以减少训练期间的模态差异。我们研究了两种特征对齐策略。Pre-AFA TSDA-Track在Transformer的模板-搜索交互之前应用对抗性对齐，以抑制模态特定的偏差。Enc-CFA TSDA-Track在交互之后对编码器表示应用对比对齐，以加强目标级别的跨模态对应。两种变体都保留了一个共享的推理流程，没有模态特定的分支。在LasHeR上的实验，以及在RGBT234和GTOT上的多种跨模态协议下的零样本评估，表明相对于代表性的最先进跟踪器有所改进。例如，在RGBT234上的模态切换协议下，Pre-AFA TSDA-Track实现了43.2/56.0的SR/PR，而ToMP-101基线为36.8/50.0。此外，对Anti-UAV-024的研究进一步验证了TSDA-Track在航空跟踪中的适用性。我们的研究强调了特征对齐域自适应对跨模态跟踪的有效性。

英文摘要

Visual object tracking typically assumes that the initial template and subsequent search frames share the same sensing modality. In practice, sensor availability or operation may change over time, creating a substantial representation gap between template and search frames. Unlike conventional multi-modal tracking where paired modalities are simultaneously available, cross-modal tracking requires localization when template and search frames originate from different active modalities. Accordingly, we introduce TSDA-Track, a Template-Search Domain Adaptation framework to reduce modality discrepancy during training. We investigate two feature alignment strategies. Pre-AFA TSDA-Track applies adversarial alignment before transformer's template-search interaction to suppress modality-specific bias. Enc-CFA TSDA-Track applies contrastive alignment to encoder representations after interaction to strengthen target-level cross-modal correspondence. Both variants retain a shared inference pipeline without modality-specific branches. Experiments on LasHeR, and zero-shot evaluations on RGBT234 and GTOT under multiple cross-modal protocols demonstrate improvements over representative state-of-the-art trackers. For instance, under the modality-switch protocol on RGBT234, Pre-AFA TSDA-Track achieves an SR/PR of 43.2/56.0, compared with 36.8/50.0 for ToMP-101 baseline. In addition, a study on Anti-UAV-024 further verifies the applicability of TSDA-Track to aerial tracking. Our study highlights the effectiveness of feature alignment domain adaptation for cross-modal tracking.

发表机构

• University of Victoria（维多利亚大学）

机构由 AI 辅助整理，请以论文原文为准。