InsEdit：通过数据高效的视频扩散模型适应实现基于指令的视觉编辑
InsEdit: Towards Instruction-based Visual Editing via Data-Efficient Video Diffusion Models Adaptation
AI总结：
本文提出InsEdit，基于HunyuanVideo-1.5构建指令式编辑模型，结合Mutual Context Attention机制，仅需少量视频编辑数据即可实现高质量视频编辑，并支持图像编辑。
摘要：
基于文本的视频编辑是一种通过文本控制视频内容的自然方法，但将视频生成模型适应为编辑器通常需要大量数据。同时，高质量视频编辑数据仍然稀缺。本文表明，一个视频生成骨干网络可以在没有大规模视频编辑数据的情况下成为强大的视频编辑器。我们提出了InsEdit，一个基于HunyuanVideo-1.5的指令式编辑模型。InsEdit结合了视觉编辑架构和基于Mutual Context Attention（MCA）的视频数据管道，创建了对齐的视频对，使得可以在剪辑中间开始编辑，而不仅仅从第一帧开始。仅需O(100)K视频编辑数据，InsEdit在我们的视频指令编辑基准上实现了最先进的开源方法结果。此外，由于我们的训练配方还包括图像编辑数据，最终模型支持图像编辑而无需任何修改。
英文摘要：
Instruction-based video editing is a natural way to control video content with text, but adapting a video generation model into an editor usually appears data-hungry. At the same time, high-quality video editing data remains scarce. In this paper, we show that a video generation backbone can become a strong video editor without large scale video editing data. We present InsEdit, an instruction-based editing model built on HunyuanVideo-1.5. InsEdit combines a visual editing architecture with a video data pipeline based on Mutual Context Attention (MCA), which creates aligned video pairs where edits can begin in the middle of a clip rather than only from the first frame. With only O(100)K video editing data, InsEdit achieves state-of-the-art results among open-source methods on our video instruction editing benchmarks. In addition, because our training recipe also includes image editing data, the final model supports image editing without any modification.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
