Cut-and-Paste：基于注意力控制的主题驱动视频编辑
Cut-and-Paste: Subject-Driven Video Editing with Attention Control
AI总结：
本文提出 Cut-and-Paste 框架，通过引入参考图像和融合相邻帧注意力图的交叉注意力控制，实现更精确的主题驱动视频编辑，在背景保留与时空一致性间取得平衡，且优于现有方法。
摘要：
本文提出了一个名为 Cut-and-Paste 的新框架，用于在文本提示和额外参考图像的指导下进行真实世界的语义视频编辑。尽管文本驱动的视频编辑在遵循给定文本提示生成高度多样化视频方面展现出了卓越的能力，但在物体细节和编辑区域方面，仅靠纯文本提示难以控制细粒度的语义编辑，且该任务通常需要冗长繁琐的文本描述。因此，我们研究了主题驱动的视频编辑，以更精确地控制编辑区域和背景保留，以及细粒度的语义生成。我们通过引入参考图像作为文本驱动视频编辑的补充输入来实现这一目标，从而避免了绞尽脑汁构思繁琐的文本提示来描述物体的详细外观。为了限制编辑区域，我们借鉴了图像编辑中的交叉注意力控制方法，并通过融合相邻帧的注意力图将其成功扩展到视频编辑中，这在保持视频背景和时空一致性之间取得了平衡。与现有方法相比，我们方法的整个过程就像是“剪切”待编辑的源物体，然后“粘贴”参考图像提供的目标物体。我们通过定量和主观评估证明，在文本提示和额外参考图像的指导下，我们的方法在视频编辑方面优于现有技术。
英文摘要：
This paper presents a novel framework termed Cut-and-Paste for real-word semantic video editing under the guidance of text prompt and additional reference image. While the text-driven video editing has demonstrated remarkable ability to generate highly diverse videos following given text prompts, the fine-grained semantic edits are hard to control by plain textual prompt only in terms of object details and edited region, and cumbersome long text descriptions are usually needed for the task. We therefore investigate subject-driven video editing for more precise control of both edited regions and background preservation, and fine-grained semantic generation. We achieve this goal by introducing an reference image as supplementary input to the text-driven video editing, which avoids racking your brain to come up with a cumbersome text prompt describing the detailed appearance of the object. To limit the editing area, we refer to a method of cross attention control in image editing and successfully extend it to video editing by fusing the attention map of adjacent frames, which strikes a balance between maintaining video background and spatio-temporal consistency. Compared with current methods, the whole process of our method is like ``cut" the source object to be edited and then ``paste" the target object provided by reference image. We demonstrate that our method performs favorably over prior arts for video editing under the guidance of text prompt and extra reference image, as measured by both quantitative and subjective evaluations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
