多模态片段组合网络：基于重要性-连贯性奖励的广告视频编辑
Multi-modal Segment Assemblage Network for Ad Video Editing with Importance-Coherence Reward
AI总结：
针对广告视频编辑中片段组合阶段依赖繁琐模型且性能不佳的问题，提出多模态片段组合网络M-SAN，采用编码器-解码器Ptr-Net框架和重要性-连贯性奖励，在Ads-1k数据集上优于现有方法。
摘要：
广告视频编辑旨在将广告视频自动剪辑为更短的视频，同时保留连贯的内容和广告商传达的关键信息。该任务主要包含两个阶段：视频分割和片段组合。现有方法在视频分割阶段表现良好，但在片段组合阶段存在依赖额外繁琐模型且性能不佳的问题。为解决这些问题，我们提出了M-SAN（多模态片段组合网络），它能够端到端地执行高效且连贯的片段组合任务。该方法利用从片段中提取的多模态表示，并采用带有注意力机制的编码器-解码器Ptr-Net框架。我们设计了重要性-连贯性奖励来训练M-SAN。我们在Ads-1k数据集上进行了实验，该数据集包含从广告商收集的1000多个视频，覆盖丰富的广告场景。为评估方法，我们提出了一个统一指标Imp-Coh@Time，该指标同时综合评估输出的重要性、连贯性和时长。实验结果表明，在该指标上，我们的方法优于随机选择和先前的方法。消融实验进一步验证了多模态表示和重要性-连贯性奖励显著提升了性能。Ads-1k数据集可在以下网址获取：https://github.com/yunlong10/Ads-1k
英文摘要：
Advertisement video editing aims to automatically edit advertising videos into shorter videos while retaining coherent content and crucial information conveyed by advertisers. It mainly contains two stages: video segmentation and segment assemblage. The existing method performs well at video segmentation stages but suffers from the problems of dependencies on extra cumbersome models and poor performance at the segment assemblage stage. To address these problems, we propose M-SAN (Multi-modal Segment Assemblage Network) which can perform efficient and coherent segment assemblage task end-to-end. It utilizes multi-modal representation extracted from the segments and follows the Encoder-Decoder Ptr-Net framework with the Attention mechanism. Importance-coherence reward is designed for training M-SAN. We experiment on the Ads-1k dataset with 1000+ videos under rich ad scenarios collected from advertisers. To evaluate the methods, we propose a unified metric, Imp-Coh@Time, which comprehensively assesses the importance, coherence, and duration of the outputs at the same time. Experimental results show that our method achieves better performance than random selection and the previous method on the metric. Ablation experiments further verify that multi-modal representation and importance-coherence reward significantly improve the performance. Ads-1k dataset is available at: https://github.com/yunlong10/Ads-1k
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
