arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2004.03661cs.IRcs.CLcs.CV
收藏

查询可控的视频摘要

Query-controllable Video Summarization

University of Amsterdam(阿姆斯特丹大学)

Jia-Hong Huang, Marcel Worring

摘要:

当视频集合变得庞大时,如何有效地在视频内和视频间进行探索是一个挑战。视频摘要是解决这个问题的方法之一。传统的摘要方法限制了视频探索的有效性,因为它们仅为给定的输入视频生成一个固定的视频摘要,与用户的信息需求无关。在这项工作中,我们介绍了一种方法,该方法将基于文本的查询作为输入,并生成与之对应的视频摘要。我们通过将视频摘要建模为监督学习问题来实现这一点,并提出了一种基于端到端深度学习的查询可控视频摘要方法,以生成查询相关的视频摘要。我们提出的方法包括一个视频摘要控制器,视频摘要生成器,和视频摘要输出模块。为了促进查询可控的视频摘要的研究和进行我们的实验,我们引入了一个数据集,其中包含基于帧的相关性分数标签。实验结果表明,基于文本的查询有助于控制视频摘要。它还表明基于文本的查询提高了我们的模型的性能。我们的代码和数据集:https://github.com/Jhhuangkay/Query-controllable-Video-Summarization。

英文摘要:

When video collections become huge, how to explore both within and across videos efficiently is challenging. Video summarization is one of the ways to tackle this issue. Traditional summarization approaches limit the effectiveness of video exploration because they only generate one fixed video summary for a given input video independent of the information need of the user. In this work, we introduce a method which takes a text-based query as input and generates a video summary corresponding to it. We do so by modeling video summarization as a supervised learning problem and propose an end-to-end deep learning based method for query-controllable video summarization to generate a query-dependent video summary. Our proposed method consists of a video summary controller, video summary generator, and video summary output module. To foster the research of query-controllable video summarization and conduct our experiments, we introduce a dataset that contains frame-based relevance score labels. Based on our experimental result, it shows that the text-based query helps control the video summary. It also shows the text-based query improves our model performance. Our code and dataset: https://github.com/Jhhuangkay/Query-controllable-Video-Summarization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑