arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.01686cs.CV
收藏

ODTrack:用于视觉跟踪的在线密集时序Token学习

ODTrack: Online Dense Temporal Token Learning for Visual Tracking

Guangxi Normal University(广西师范大学);Wuzhou University(梧州学院);Harbin Institute of Technology(哈尔滨工业大学)

Yaozong Zheng, Bineng Zhong, Qihua Liang, Zhiyi Mo, Shengping Zhang, Xianxian Li

AI总结:

ODTrack提出一种视频级跟踪流水线,通过在线token传播密集关联视频帧上下文,将目标判别特征压缩为token序列实现帧间关联,在七个基准上达到SOTA性能且实时运行。

摘要:

连续视频帧之间的在线上下文推理和关联对于视觉跟踪中感知实例至关重要。然而,目前大多数表现最优的跟踪器仍然通过离线模式依赖于参考帧和搜索帧之间的稀疏时序关系。因此,它们只能在每个图像对内独立交互,并建立有限的时序关联。为了缓解上述问题,我们提出了一种简单、灵活且有效的视频级跟踪流水线,命名为ODTrack,它以在线token传播的方式密集关联视频帧的上下文关系。ODTrack接收任意长度的视频帧以捕获实例的时空轨迹关系,并将目标的判别特征(定位信息)压缩为token序列以实现帧到帧的关联。这种新方案带来以下优势:1)纯化的token序列可作为下一视频帧推理的提示,从而利用过去信息指导未来推理;2)通过token序列的迭代传播,有效避免了复杂的在线更新策略,从而实现更高效的模型表示和计算。ODTrack在七个基准上实现了新的SOTA性能,同时以实时速度运行。代码和模型可在https://github.com/GXNU-ZhongLab/ODTrack获取。

英文摘要:

Online contextual reasoning and association across consecutive video frames are critical to perceive instances in visual tracking. However, most current top-performing trackers persistently lean on sparse temporal relationships between reference and search frames via an offline mode. Consequently, they can only interact independently within each image-pair and establish limited temporal correlations. To alleviate the above problem, we propose a simple, flexible and effective video-level tracking pipeline, named \textbf{ODTrack}, which densely associates the contextual relationships of video frames in an online token propagation manner. ODTrack receives video frames of arbitrary length to capture the spatio-temporal trajectory relationships of an instance, and compresses the discrimination features (localization information) of a target into a token sequence to achieve frame-to-frame association. This new solution brings the following benefits: 1) the purified token sequences can serve as prompts for the inference in the next video frame, whereby past information is leveraged to guide future inference; 2) the complex online update strategies are effectively avoided by the iterative propagation of token sequences, and thus we can achieve more efficient model representation and computation. ODTrack achieves a new \textit{SOTA} performance on seven benchmarks, while running at real-time speed. Code and models are available at \url{https://github.com/GXNU-ZhongLab/ODTrack}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑