arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.09598cs.CV
收藏

基于用户历史学习的自适应视频高光检测

Adaptive Video Highlight Detection by Learning from User History

University of Manitoba(曼尼托巴大学);Huawei Technologies(华为技术有限公司)

Mrigank Rochan, Mahesh Kumar Krishna Reddy, Linwei Ye, Yang Wang

摘要:

最近,有越来越多的兴趣在亮点检测研究,其目标是创建一个短的持续时间的视频从较长的视频提取其有趣的时刻。然而,大多数现有的方法忽略了这样一个事实,即视频亮点的定义是高度主观的。不同的用户对于相同的输入视频可能具有不同的高光偏好。在本文中,我们提出了一个简单而有效的框架,学习适应突出检测到用户利用用户的历史,突出显示的形式,用户以前创建的。我们的框架由两个子网络组成:一个完全时间卷积的亮点检测网络$H$,预测输入视频的亮点和历史编码器网络$M$的用户历史。我们引入了一个新设计的时间自适应实例规范化(T-AIN)层的$H$的两个子网络相互作用。T-AIN具有基于用户历史从$M$预测的仿射参数,并且负责到$H$的用户自适应信号。在大规模数据集上的大量实验表明,我们的框架可以做出更准确和用户特定的亮点预测。

英文摘要:

Recently, there is an increasing interest in highlight detection research where the goal is to create a short duration video from a longer video by extracting its interesting moments. However, most existing methods ignore the fact that the definition of video highlight is highly subjective. Different users may have different preferences of highlight for the same input video. In this paper, we propose a simple yet effective framework that learns to adapt highlight detection to a user by exploiting the user's history in the form of highlights that the user has previously created. Our framework consists of two sub-networks: a fully temporal convolutional highlight detection network $H$ that predicts highlight for an input video and a history encoder network $M$ for user history. We introduce a newly designed temporal-adaptive instance normalization (T-AIN) layer to $H$ where the two sub-networks interact with each other. T-AIN has affine parameters that are predicted from $M$ based on the user history and is responsible for the user-adaptive signal to $H$. Extensive experiments on a large-scale dataset show that our framework can make more accurate and user-specific highlight predictions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑