EffiVED：基于文本指令扩散模型的高效视频编辑
EffiVED:Efficient Video Editing via Text-instruction Diffusion Models
AI总结：
本文提出基于扩散模型的高效视频编辑模型EffiVED，通过两种工作流收集数据构建高质量训练集，实现无需逐视频微调的指令引导视频编辑，在保证高质量的同时实现快速执行。
摘要：
大规模文本到视频模型已展现出卓越的能力，但由于可用数据集有限，其直接应用于视频编辑仍具挑战。当前的视频编辑方法通常需要对扩散模型进行逐视频微调或特定的反演优化，以确保高保真编辑。本文提出了一种高效且基于扩散的模型EffiVED，可直接支持指令引导的视频编辑。为此，我们提出了两种高效的工作流，利用数据增强和基础视觉语言技术来收集视频编辑对。这些工作流将海量图像编辑数据集和开放世界视频转化为训练EffiVED的高质量数据集。实验结果表明，EffiVED不仅能生成高质量的编辑视频，而且执行迅速。最后，我们证明我们的数据收集方法显著提升了编辑性能，并有望解决视频编辑数据匮乏的问题。代码可在https://github.com/alibaba/EffiVED找到。
英文摘要：
Large-scale text-to-video models have shown remarkable abilities, but their direct application in video editing remains challenging due to limited available datasets. Current video editing methods commonly require per-video fine-tuning of diffusion models or specific inversion optimization to ensure high-fidelity edits. In this paper, we introduce EffiVED, an efficient diffusion-based model that directly supports instruction-guided video editing. To achieve this, we present two efficient workflows to gather video editing pairs, utilizing augmentation and fundamental vision-language techniques. These workflows transform vast image editing datasets and open-world videos into a high-quality dataset for training EffiVED. Experimental results reveal that EffiVED not only generates high-quality editing videos but also executes rapidly. Finally, we demonstrate that our data collection method significantly improves editing performance and can potentially tackle the scarcity of video editing data. Code can be found at https://github.com/alibaba/EffiVED.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
