arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.16469cs.CV
收藏

基于时空Transformer的视频身份识别

Spatiotemporal Transformer for Video-based Person Re-identification

Beihang University(北京航空航天大学);Tsinghua University(清华大学);University of Science and Technology of China(中国科学技术大学);Pengcheng Laboratory(鹏城实验室);Xidian University(西安电子科技大学)

Tianyu Zhang, Longhui Wei, Lingxi Xie, Zijie Zhuang, Yongfei Zhang, Bo Li, Qi Tian

摘要:

最近,Transformer模块已经从自然语言处理移植到计算机视觉。本文将Transformer应用于基于视频的人物再识别中,其关键问题是从一段轨迹中提取具有鉴别力的信息。我们发现,尽管有很强的学习能力,香草Transformer遭受过度拟合的风险增加,可以说是由于大量的注意力参数和不足的训练数据。为了解决这个问题,我们提出了一种新的管道,其中模型在一组合成的视频数据上进行预训练,然后使用感知约束的时空Transformer(STT)模块和全局Transformer(GT)模块将其转移到下游域。衍生算法在三个流行的基于视频的人员重新识别基准,MARS,DukeMTMC-VideoReID和LS-VID上实现了显着的准确性增益,特别是当训练和测试数据来自不同的域时。更重要的是,我们的研究揭示了Transformer在高度结构化的视觉数据上的应用。

英文摘要:

Recently, the Transformer module has been transplanted from natural language processing to computer vision. This paper applies the Transformer to video-based person re-identification, where the key issue is to extract the discriminative information from a tracklet. We show that, despite the strong learning ability, the vanilla Transformer suffers from an increased risk of over-fitting, arguably due to a large number of attention parameters and insufficient training data. To solve this problem, we propose a novel pipeline where the model is pre-trained on a set of synthesized video data and then transferred to the downstream domains with the perception-constrained Spatiotemporal Transformer (STT) module and Global Transformer (GT) module. The derived algorithm achieves significant accuracy gain on three popular video-based person re-identification benchmarks, MARS, DukeMTMC-VideoReID, and LS-VID, especially when the training and testing data are from different domains. More importantly, our research sheds light on the application of the Transformer on highly-structured visual data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑