Goku：百万级通用指令视频编辑数据集与基准
Goku: A Million-Scale Universal Dataset and Benchmark for Instruction-Based Video Editing
AI总结：
提出首个百万级多任务视频编辑数据集Goku，包含200万高质量指令对齐对，并设计高效数据合成流程和渐进过滤系统，基于此提出双分支模型Goku-Edit及包含1000个测试用例的基准Goku-Bench，在指令遵循上提升8%。
摘要：
现有的基于指令的视频编辑数据集通常专注于单任务外观编辑，无法满足现实场景中复杂的创意需求。为弥补这一差距，我们提出Goku，一个大规模数据集，包含200万高质量、指令对齐的视频编辑对，这是首个将任务边界从基本外观编辑扩展到多任务和结构操作（例如，主体运动的精确控制）的数据集。为了应对这些复杂任务中固有的数据合成挑战，我们设计了一个高效的数据合成流程，将复杂编辑分解为可控的子问题，并在整个过程中引入渐进过滤系统以确保数据可靠性。此外，我们在Goku上探索了最优网络结构，并提出了Goku-Edit。为了深入理解复杂的编辑指令，Goku-Edit采用MLLM作为其文本编码器，并采用解耦的双分支设计：专用掩码分支处理结构控制，释放主分支进行外观渲染。还提出了一个全面的视频编辑基准Goku-Bench，包含1000个人工验证的测试用例和7个新颖的编辑特定指标。在Goku-Bench上评估，Goku-Edit在指令遵循方面比其他开源模型提升了高达8%。
英文摘要：
Existing instruction-based video editing datasets commonly focus on single-task appearance editing, failing to meet the complex creative demands of real-world scenarios. To bridge this gap, we present Goku, a large-scale dataset featuring 2 million high-quality, instruction-aligned video editing pairs, which is the first to extend task boundaries from basic appearance editing to multi-task and structural manipulations(e.g., precise control of subject movement). To tackle the data synthesis challenges inherent in these complex tasks, we design an efficient data synthesis pipeline that decomposes complex edits into controllable sub-problems and introduce a progressive filtering system for data reliability throughout the whole process. Furthermore, we explore the optimal network structures on Goku, and propose Goku-Edit. To deeply comprehend complex editing instructions, Goku-Edit leverages an MLLM as its text encoder and adopts a decoupled dual-branch design: a dedicated mask branch handles structural control, freeing the main branch for appearance rendering. A comprehensive video editing benchmark, Goku-Bench, is also proposed with 1,000 human-verified test cases and 7 novel editing-specific metrics. Evaluated on Goku-Bench, Goku-Edit obtains up to +8% improvement on other open-source models in terms of instruction following.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
