最短片段，最大显著性：通过关键时刻提取实现长视频摘要
Minimal Clips, Maximum Salience: Long Video Summarization via Key Moment Extraction
AI总结：
本文提出了一种通过关键时刻提取实现长视频多模态摘要的方法，利用轻量级模型和大型语言模型实现高效摘要生成。
摘要：
视觉-语言模型（VLMs）能够处理越来越长的视频。然而，重要的视觉信息在整个上下文中容易丢失，并被VLMs所忽略。此外，设计能够有效分析长视频内容的工具也很重要。在本文中，我们提出了一种片段选择方法，旨在将关键视频时刻包含在多模态摘要中。我们将视频分成短片段，并使用轻量级视频描述模型为每个片段生成紧凑的视觉描述。这些描述随后传递给一个大型语言模型（LLM），该模型选择包含最多相关视觉信息的K个片段以构建多模态摘要。我们在MovieSum数据集上评估了我们的方法，该数据集的参考片段是自动从完整的人工标注剧本和摘要中推导出来的。我们进一步表明，这些参考片段（少于6%的电影）足以构建MovieSum中电影的完整多模态摘要。使用我们的片段选择方法，我们实现了接近参考片段的摘要性能，同时捕捉到比随机片段选择显著更多的相关视频信息。重要的是，我们通过依赖轻量级描述模型保持了低计算成本。
英文摘要：
Vision-Language Models (VLMs) are able to process increasingly longer videos. Yet, important visual information is easily lost throughout the entire context and missed by VLMs. Also, it is important to design tools that enable cost-effective analysis of lengthy video content. In this paper, we propose a clip selection method that targets key video moments to be included in a multimodal summary. We divide the video into short clips and generate compact visual descriptions of each using a lightweight video captioning model. These are then passed to a large language model (LLM), which selects the K clips containing the most relevant visual information for a multimodal summary. We evaluate our approach on reference clips for the task, automatically derived from full human-annotated screenplays and summaries in the MovieSum dataset. We further show that these reference clips (less than 6% of the movie) are sufficient to build a complete multimodal summary of the movies in MovieSum. Using our clip selection method, we achieve a summarization performance close to that of these reference clips while capturing substantially more relevant video information than random clip selection. Importantly, we maintain low computational cost by relying on a lightweight captioning model.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
