Kiwi-Edit：通过指令和参考引导实现多功能视频编辑
Kiwi-Edit: Versatile Video Editing via Instruction and Reference Guidance
AI总结：
本文提出Kiwi-Edit框架，通过指令和参考引导实现可控视频编辑，构建了RefVIE数据集和评估基准，并通过多阶段训练提升指令遵循和参考保真度。
摘要：
基于指令的视频编辑技术取得了快速发展，但现有方法在精确视觉控制上面临挑战，因自然语言难以描述复杂视觉细节。尽管参考引导编辑提供了稳健解决方案，但其潜力受限于高质量配对训练数据的稀缺。为此，我们引入可扩展的数据生成管道，将现有视频编辑配对转换为高保真训练四元组，利用图像生成模型创建合成参考框架。通过该管道，我们构建了专为指令-参考跟随任务设计的RefVIE大规模数据集，并建立了RefVIE-Bench进行综合评估。此外，我们提出了统一的编辑架构Kiwi-Edit，通过可学习查询和潜在视觉特征实现参考语义引导。我们的模型通过渐进多阶段训练课程，在指令遵循和参考保真度上取得显著提升。大量实验表明，我们的数据和架构在可控视频编辑中建立了新的最先进水平。所有数据集、模型和代码已发布在https://github.com/showlab/Kiwi-Edit。
英文摘要：
Instruction-based video editing has witnessed rapid progress, yet current methods often struggle with precise visual control, as natural language is inherently limited in describing complex visual nuances. Although reference-guided editing offers a robust solution, its potential is currently bottlenecked by the scarcity of high-quality paired training data. To bridge this gap, we introduce a scalable data generation pipeline that transforms existing video editing pairs into high-fidelity training quadruplets, leveraging image generative models to create synthesized reference scaffolds. Using this pipeline, we construct RefVIE, a large-scale dataset tailored for instruction-reference-following tasks, and establish RefVIE-Bench for comprehensive evaluation. Furthermore, we propose a unified editing architecture, Kiwi-Edit, that synergizes learnable queries and latent visual features for reference semantic guidance. Our model achieves significant gains in instruction following and reference fidelity via a progressive multi-stage training curriculum. Extensive experiments demonstrate that our data and architecture establish a new state-of-the-art in controllable video editing. All datasets, models, and code is released at https://github.com/showlab/Kiwi-Edit.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
