arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2109.01799cs.CV
收藏

PR-Net:用于个性化视频高光检测的偏好推理

PR-Net: Preference Reasoning for Personalized Video Highlight Detection

The University of Hong Kong(香港大学);Youtu Lab, Tencent(腾讯优图实验室);Zhejiang University(浙江大学)

Runnan Chen, Penghao Zhou, Wenzhe Wang, Nenglun Chen, Pai Peng, Xing Sun, Wenping Wang

摘要:

个性化视频精彩片段检测旨在根据用户的偏好将长视频缩短为有趣的时刻,这最近引起了社区的关注。目前的方法把用户的历史作为整体信息来预测用户的偏好,但否定了用户的兴趣的固有多样性,导致模糊的偏好表示。在本文中,我们提出了一个简单而有效的偏好推理框架(PR-Net),明确考虑到不同的利益,帧级的亮点预测。具体地,针对每个输入查询帧产生不同的用户特定偏好,呈现为历史突出显示与对应查询帧的相似性加权和。接下来,由用户特定的偏好和可学习的通用偏好形成不同的综合偏好,以进行更全面的高光测量。最后,计算每个查询帧的高亮和非高亮程度,分别作为其综合和非高亮偏好的语义相似度。此外,为了减轻由于不完整的注释,一个新的双向对比损失,以确保一个紧凑的和可微的度量空间。通过这种方式,我们的方法显着优于最先进的方法,平均准确度相对提高了12%。

英文摘要:

Personalized video highlight detection aims to shorten a long video to interesting moments according to a user's preference, which has recently raised the community's attention. Current methods regard the user's history as holistic information to predict the user's preference but negating the inherent diversity of the user's interests, resulting in vague preference representation. In this paper, we propose a simple yet efficient preference reasoning framework (PR-Net) to explicitly take the diverse interests into account for frame-level highlight prediction. Specifically, distinct user-specific preferences for each input query frame are produced, presented as the similarity weighted sum of history highlights to the corresponding query frame. Next, distinct comprehensive preferences are formed by the user-specific preferences and a learnable generic preference for more overall highlight measurement. Lastly, the degree of highlight and non-highlight for each query frame is calculated as semantic similarity to its comprehensive and non-highlight preferences, respectively. Besides, to alleviate the ambiguity due to the incomplete annotation, a new bi-directional contrastive loss is proposed to ensure a compact and differentiable metric space. In this way, our method significantly outperforms state-of-the-art methods with a relative improvement of 12% in mean accuracy precision.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑