面向视频片段蒙太奇的文本-视频多粒度融合方法
Text-Video Multi-Grained Integration for Video Moment Montage
AI总结：
针对短视频手动编辑效率低的问题，提出文本-视频多粒度融合方法TV-MGI，实现视频与脚本的全局及细粒度对齐，同时构建专用大规模数据集MSSD，实验验证其效果优于基线方法。
摘要：
在线短视频平台的蓬勃发展使用户对短视频编辑的需求急剧增长。然而，手动选择、裁剪原始素材并将其组装成连贯且高质量的视频仍然十分费力且耗时。为加快这一流程，我们聚焦于一项用户友好的新任务——视频片段蒙太奇（Video Moment Montage，VMM），该任务旨在根据预先提供的旁白文本准确定位对应的视频片段，随后编排这些视频片段以生成与对应描述相符的完整视频。该任务的难点在于，单个脚本句子可能需要修剪并组装多个视频片段，因此既要提取精确的时间片段，又要确保句内和句间的上下文一致性。为解决这一问题，我们提出了一种新颖的文本-视频多粒度融合方法（Text-Video Multi-Grained Integration，TV-MGI），该方法可高效融合脚本的文本特征与镜头级、帧级的视频特征，实现视频内容与脚本中对应文本描述的全局及细粒度对齐。为推动该领域的进一步研究，我们构建了多句子镜头数据集（Multiple Sentences with Shots Dataset，MSSD），这是一个专门为VMM任务设计的大规模数据集。我们在MSSD数据集上开展了大量实验，结果表明，与基线方法相比，我们的框架具备更优的有效性。
英文摘要：
The proliferation of online short video platforms has driven a surge in user demand for short video editing. However, manually selecting, cropping, and assembling raw footage into a coherent, high-quality video remains laborious and time-consuming. To accelerate this process, we focus on a user-friendly new task called Video Moment Montage (VMM), which aims to accurately locate the corresponding video segments based on a pre-provided narration text and then arrange these video clips to create a complete video that aligns with the corresponding descriptions. The challenge lies in extracting precise temporal segments while ensuring intra-sentence and inter-sentence context consistency, as a single script sentence may require trimming and assembling multiple video clips. To address this problem, we present a novel \textit{Text-Video Multi-Grained Integration} method (TV-MGI) that efficiently fuses text features from the script with both shot-level and frame-level video features, which enables the global and fine-grained alignment between the video content and the corresponding textual descriptions in the script. To facilitate further research in this area, we introduce the Multiple Sentences with Shots Dataset (MSSD), a large-scale dataset designed explicitly for the VMM task. We conduct extensive experiments on the MSSD dataset to demonstrate the effectiveness of our framework compared to baseline methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
