arXiv 2502.067342025-03-12cs.CV

Señorita-2M：由视频专家构建的面向通用视频编辑的高质量指令数据集

Señorita-2M: A High-Quality Instruction-based Dataset for General Video Editing by Video Specialists

The Chinese University of Hong Kong（香港中文大学）；The Hong Kong Polytechnic University（香港理工大学）；Tsinghua University（清华大学）；The University of Hong Kong（香港大学）；IntelliFusion Inc.（智融公司）；University of Electronic Science and Technology of China（电子科技大学）

Bojia Zi, Penghui Ruan, Marco Chen, Xianbiao Qi, Shaozhe Hao, Shihao Zhao, Youze Huang, Bin Liang, Rong Xiao, Kam-Fai Wong

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出包含约200万个高质量视频编辑对的Señorita-2M数据集，通过四个专用编辑模型构建并设计过滤流程，以弥补端到端视频编辑方法训练数据不足的问题，实验证明其能显著提升视频编辑质量。

摘要：

视频生成领域的最新进展推动了视频编辑技术的发展，这些技术可分为基于反演的方法和端到端方法。然而，当前的视频编辑方法仍面临若干挑战。基于反演的方法虽然无需训练且灵活，但在推理过程中耗时较长，难以处理细粒度编辑指令，并且会产生伪影和抖动。另一方面，端到端方法依赖编辑视频对进行训练，推理速度更快，但由于缺乏高质量的训练视频对，往往产生较差的编辑结果。在本文中，为了弥补端到端方法的不足，我们提出了Señorita-2M，一个高质量的视频编辑数据集。Señorita-2M包含约200万个视频编辑对。该数据集通过构建四个高质量、专用的视频编辑模型而建立，每个模型均由我们团队精心构建和训练，以实现最先进的编辑效果。我们还提出了一种过滤流程，以剔除编辑质量较差的视频对。此外，我们探索了常见的视频编辑架构，以基于当前预训练生成模型确定最有效的结构。大量实验表明，我们的数据集有助于产生显著高质量的视频编辑结果。更多详情请访问https://senorita-2m-dataset.github.io。

英文摘要：

Recent advancements in video generation have spurred the development of video editing techniques, which can be divided into inversion-based and end-to-end methods. However, current video editing methods still suffer from several challenges. Inversion-based methods, though training-free and flexible, are time-consuming during inference, struggle with fine-grained editing instructions, and produce artifacts and jitter. On the other hand, end-to-end methods, which rely on edited video pairs for training, offer faster inference speeds but often produce poor editing results due to a lack of high-quality training video pairs. In this paper, to close the gap in end-to-end methods, we introduce Señorita-2M, a high-quality video editing dataset. Señorita-2M consists of approximately 2 millions of video editing pairs. It is built by crafting four high-quality, specialized video editing models, each crafted and trained by our team to achieve state-of-the-art editing results. We also propose a filtering pipeline to eliminate poorly edited video pairs. Furthermore, we explore common video editing architectures to identify the most effective structure based on current pre-trained generative model. Extensive experiments show that our dataset can help to yield remarkably high-quality video editing results. More details are available at https://senorita-2m-dataset.github.io.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。