arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.10043cs.CVcs.LGcs.MM
收藏

利用门控多级注意和时间对抗训练增强Transformer的视频理解

Enhancing Transformer for Video Understanding Using Gated Multi-Level Attention and Temporal Adversarial Training

Samsung Research America(美国三星研究院)

Saurabh Sahu, Palash Goyal

摘要:

Transformer模型的引入使序列建模,特别是文本领域的建模取得了巨大的进步。然而,使用基于注意力的模型进行视频理解仍然是相对未开发的。在本文中,我们引入了门控对抗Transformer(GAT),以增强基于注意力的模型对视频的适用性。GAT使用多级注意门来基于局部和全局上下文对帧的相关性进行建模。这使得模型能够以各种粒度理解视频。此外,GAT使用对抗训练来提高模型的泛化能力。我们提出了时间注意力正则化方案,以提高注意力模块对对抗性示例的鲁棒性。我们说明了GAT在大规模YoutTube-8 M数据集上的视频分类任务的性能。我们进一步显示消融研究以及定量和定性分析,以展示改善。

英文摘要:

The introduction of Transformer model has led to tremendous advancements in sequence modeling, especially in text domain. However, the use of attention-based models for video understanding is still relatively unexplored. In this paper, we introduce Gated Adversarial Transformer (GAT) to enhance the applicability of attention-based models to videos. GAT uses a multi-level attention gate to model the relevance of a frame based on local and global contexts. This enables the model to understand the video at various granularities. Further, GAT uses adversarial training to improve model generalization. We propose temporal attention regularization scheme to improve the robustness of attention modules to adversarial examples. We illustrate the performance of GAT on the large-scale YoutTube-8M data set on the task of video categorization. We further show ablation studies along with quantitative and qualitative analysis to showcase the improvement.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑