个性化视频摘要：基于多模态视频理解
Personalized Video Summarization by Multimodal Video Understanding
AI总结：
本文提出基于预训练视觉语言模型的用户偏好视频摘要流水线VSL，通过场景级语义分析选择相关文本场景，无需大规模训练数据，实验表明其优于现有无监督模型且更具适应性。
摘要：
视频摘要技术已被证明能够改善用户在访问和理解视频内容时的整体体验。如果已知用户的偏好，视频摘要可以从输入视频中识别出重要信息或相关内容，帮助用户获取必要信息或判断是否对观看原视频感兴趣。将视频摘要适配到不同类型的视频和用户偏好，需要大量的训练数据和昂贵的人工标注。为了促进此类研究，我们提出了一个新的视频摘要基准，用于捕捉各种用户偏好。此外，我们提出了一种名为“带语言的视频摘要”（VSL）的流水线，用于用户偏好的视频摘要，该流水线基于预训练的视觉语言模型（VLM），以避免在大型训练数据集上训练视频摘要系统的需求。该流水线将视频和隐藏字幕作为输入，并通过将视频帧转换为文本，在场景级别进行语义分析。随后，以用户的类型偏好为基础，选择相关的文本场景。实验结果表明，我们提出的流水线优于当前最先进的无监督视频摘要模型。我们展示了我们的方法在不同数据集上的适应性优于基于监督查询的视频摘要模型。最后，运行时分析表明，当用户偏好和视频数量增加时，我们的流水线更适合实际应用。
英文摘要：
Video summarization techniques have been proven to improve the overall user experience when it comes to accessing and comprehending video content. If the user's preference is known, video summarization can identify significant information or relevant content from an input video, aiding them in obtaining the necessary information or determining their interest in watching the original video. Adapting video summarization to various types of video and user preferences requires significant training data and expensive human labeling. To facilitate such research, we proposed a new benchmark for video summarization that captures various user preferences. Also, we present a pipeline called Video Summarization with Language (VSL) for user-preferred video summarization that is based on pre-trained visual language models (VLMs) to avoid the need to train a video summarization system on a large training dataset. The pipeline takes both video and closed captioning as input and performs semantic analysis at the scene level by converting video frames into text. Subsequently, the user's genre preference was used as the basis for selecting the pertinent textual scenes. The experimental results demonstrate that our proposed pipeline outperforms current state-of-the-art unsupervised video summarization models. We show that our method is more adaptable across different datasets compared to supervised query-based video summarization models. In the end, the runtime analysis demonstrates that our pipeline is more suitable for practical use when scaling up the number of user preferences and videos.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
