arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2008.07728cs.CV
收藏

弱监督时间动作局部化的等价分类映射

Equivalent Classification Mapping for Weakly Supervised Temporal Action Localization

Northwestern Polytechnical University(西北工业大学);Xidian University(西安电子科技大学)

Tao Zhao, Junwei Han, Le Yang, Dingwen Zhang

摘要:

弱监督时间动作定位是近年来出现的一个新的研究课题。现有的方法可以分为两个分类定位流水线,即,预分类流水线和后分类流水线。预分类流水线首先对每个视频片段执行分类,然后聚合片段级分类分数以获得视频级分类分数。相比之下,后分类流水线首先聚合片段级特征,然后基于聚合的特征预测视频级分类分数。虽然这两条管道中的分类器的使用方式不同,但它们所起的作用是完全相同的-对给定的特征进行分类,以识别相应的动作类别。为此,理想的分类器可以使两个管道都工作。这启发我们在统一的框架中同时学习这两个管道,以获得有效的分类器。具体来说,在所提出的学习框架中,我们实现了两个并行的网络流,同时对两个分类定位管道进行建模,并使两个网络流共享同一个分类器。这实现了新的等效分类映射(ECM)机制。此外,我们发现,一个理想的分类器可能具有两个特性:1)从预分类流中获得的帧级分类得分和后分类流中的特征聚合权重应该是一致的; 2)这两个流的分类结果应该是相同的。基于这两个特点,我们进一步引入了权重转换模块和等价训练策略到建议的学习框架,这有助于彻底挖掘等价机制。

英文摘要:

Weakly supervised temporal action localization is a newly emerging yet widely studied topic in recent years. The existing methods can be categorized into two localization-by-classification pipelines, i.e., the pre-classification pipeline and the post-classification pipeline. The pre-classification pipeline first performs classification on each video snippet and then aggregate the snippet-level classification scores to obtain the video-level classification score. In contrast, the post-classification pipeline aggregates the snippet-level features first and then predicts the video-level classification score based on the aggregated feature. Although the classifiers in these two pipelines are used in different ways, the role they play is exactly the same---to classify the given features to identify the corresponding action categories. To this end, an ideal classifier can make both pipelines work. This inspires us to simultaneously learn these two pipelines in a unified framework to obtain an effective classifier. Specifically, in the proposed learning framework, we implement two parallel network streams to model the two localization-by-classification pipelines simultaneously and make the two network streams share the same classifier. This achieves the novel Equivalent Classification Mapping (ECM) mechanism. Moreover, we discover that an ideal classifier may possess two characteristics: 1) The frame-level classification scores obtained from the pre-classification stream and the feature aggregation weights in the post-classification stream should be consistent; 2) The classification results of these two streams should be identical. Based on these two characteristics, we further introduce a weight-transition module and an equivalent training strategy into the proposed learning framework, which assists to thoroughly mine the equivalence mechanism.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑