通过后训练增强视频生成中的场景转换意识
Enhancing Scene Transition Awareness in Video Generation via Post-Training
AI总结：
本文提出TAV数据集，通过后训练提升视频生成中场景转换的理解能力，改善多场景生成效果并保持图像质量。
摘要：
最近人工智能生成视频的进步在文本到视频任务中表现出色，尤其是在表现单一场景的短片段上。然而，当前模型在生成具有连贯场景转换的更长视频时遇到困难，主要原因是它们无法从提示中推断出何时需要转换。大多数开源模型都是在由单一场景视频片段组成的数据集上训练的，这限制了它们学习和响应需要多个场景的提示的能力。开发场景转换意识对于多场景生成至关重要，因为它使模型能够通过准确检测转换来识别和分割视频为不同的片段。为此，我们提出了Transition-Aware Video（TAV）数据集，该数据集由预处理的包含多个场景转换的视频片段组成。我们的实验表明，对TAV数据集进行后训练可以提高基于提示的场景转换理解，缩小所需场景与生成场景之间的差距，并保持图像质量。
英文摘要：
Recent advances in AI-generated video have shown strong performance on \emph{text-to-video} tasks, particularly for short clips depicting a single scene. However, current models struggle to generate longer videos with coherent scene transitions, primarily because they cannot infer when a transition is needed from the prompt. Most open-source models are trained on datasets consisting of single-scene video clips, which limits their capacity to learn and respond to prompts requiring multiple scenes. Developing scene transition awareness is essential for multi-scene generation, as it allows models to identify and segment videos into distinct clips by accurately detecting transitions. To address this, we propose the \textbf{Transition-Aware Video} (TAV) dataset, which consists of preprocessed video clips with multiple scene transitions. Our experiment shows that post-training on the \textbf{TAV} dataset improves prompt-based scene transition understanding, narrows the gap between required and generated scenes, and maintains image quality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
