arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2209.12362cs.CV
收藏

基于多数据集训练的Transformers鲁棒动作识别

Multi-dataset Training of Transformers for Robust Action Recognition

Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Tencent Youtu Lab(腾讯优图实验室);Zhejiang University(浙江大学)

Junwei Liang, Enwei Zhang, Jun Zhang, Chunhua Shen

摘要:

我们研究了鲁棒特征表示的任务,旨在在多个数据集上进行动作识别。我们建立我们的方法在Transformers的有效性。尽管在过去的十年里,我们见证了视频动作识别的巨大进步,但如何训练一个可以在多个数据集上表现良好的单一模型仍然具有挑战性和价值。在这里,我们提出了一种新的多数据集训练范式MultiTrain,设计了两个新的损失项,即信息损失和投影损失,旨在学习用于动作识别的鲁棒表示。特别是,信息损失最大化了特征嵌入的表现力,而每个数据集的投影损失挖掘了数据集之间类之间的内在关系。我们在五个具有挑战性的数据集上验证了我们的方法的有效性:Kinetics-400,Kinetics-700,Moments-in-Time,Activitynet和Something-something-v2数据集。大量的实验结果表明,我们的方法可以不断提高国家的最先进的性能。代码和模型已发布。

英文摘要:

We study the task of robust feature representations, aiming to generalize well on multiple datasets for action recognition. We build our method on Transformers for its efficacy. Although we have witnessed great progress for video action recognition in the past decade, it remains challenging yet valuable how to train a single model that can perform well across multiple datasets. Here, we propose a novel multi-dataset training paradigm, MultiTrain, with the design of two new loss terms, namely informative loss and projection loss, aiming to learn robust representations for action recognition. In particular, the informative loss maximizes the expressiveness of the feature embedding while the projection loss for each dataset mines the intrinsic relations between classes across datasets. We verify the effectiveness of our method on five challenging datasets, Kinetics-400, Kinetics-700, Moments-in-Time, Activitynet and Something-something-v2 datasets. Extensive experimental results show that our method can consistently improve state-of-the-art performance. Code and models are released.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑