arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2108.11770cs.CV
收藏

基于集合学习的跨类别视频亮点检测

Cross-category Video Highlight Detection via Set-based Learning

Shanghai Jiao Tong University(上海交通大学);ByteDance AI Lab(字节跳动人工智能实验室)

Minghao Xu, Hang Wang, Bingbing Ni, Riheng Zhu, Zhenbang Sun, Changhu Wang

摘要:

自主高光检测对于提高社交媒体平台上的视频浏览效率至关重要。为了以数据驱动的方式实现这一目标,人们可能经常面临这样的情况,即在实践中使用的目标视频类别上没有精彩注释,而对另一个视频类别(称为源视频类别)的监督是可以实现的。在这种情况下,可以通过将从源视频类别获取的精彩部分知识转移到目标视频类别来导出目标视频类别上的有效精彩部分检测器。我们称这个问题为跨类别的视频亮点检测,这在以前的作品中很少研究。为了解决这一实际问题,我们提出了一个基于双学习器的视频高光检测(DL-VHD)框架。在此框架下,我们首先设计了一个基于集合的学习模块(SL模块),以改善传统的基于对的学习,评估在更广泛的背景下的视频片段的亮点程度。基于这样的学习方式,我们引入了两个不同的学习器,分别获得目标类别视频的基本区别和源视频类别上的精彩时刻的特征。这两种类型的亮点知识通过知识蒸馏进一步巩固。在三个基准数据集上的大量实验表明了所提出的SL模块的优越性,并且DL-VHD方法在各种跨类别高光检测任务上优于五种典型的无监督领域自适应(UDA)算法。我们的代码可在https://github.com/ChrisAllenMing/Cross_Category_Video_Highlight上获得。

英文摘要:

Autonomous highlight detection is crucial for enhancing the efficiency of video browsing on social media platforms. To attain this goal in a data-driven way, one may often face the situation where highlight annotations are not available on the target video category used in practice, while the supervision on another video category (named as source video category) is achievable. In such a situation, one can derive an effective highlight detector on target video category by transferring the highlight knowledge acquired from source video category to the target one. We call this problem cross-category video highlight detection, which has been rarely studied in previous works. For tackling such practical problem, we propose a Dual-Learner-based Video Highlight Detection (DL-VHD) framework. Under this framework, we first design a Set-based Learning module (SL-module) to improve the conventional pair-based learning by assessing the highlight extent of a video segment under a broader context. Based on such learning manner, we introduce two different learners to acquire the basic distinction of target category videos and the characteristics of highlight moments on source video category, respectively. These two types of highlight knowledge are further consolidated via knowledge distillation. Extensive experiments on three benchmark datasets demonstrate the superiority of the proposed SL-module, and the DL-VHD method outperforms five typical Unsupervised Domain Adaptation (UDA) algorithms on various cross-category highlight detection tasks. Our code is available at https://github.com/ChrisAllenMing/Cross_Category_Video_Highlight .

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑