MagicProp：基于运动感知外观传播的扩散视频编辑
MagicProp: Diffusion-based Video Editing via Motion-aware Appearance Propagation
AI总结：
MagicProp提出两阶段扩散视频编辑框架，先编辑单帧外观，再通过PropDPM自回归传播，兼顾编辑灵活性与时间一致性。
摘要：
本文解决了在保持视频运动的同时修改其视觉外观的问题。我们提出了一种名为MagicProp的新框架，它将视频编辑过程分解为两个阶段：外观编辑和运动感知的外观传播。在第一阶段，MagicProp从输入视频中选择单帧，并应用图像编辑技术来修改该帧的内容和/或风格。这些技术的灵活性使得能够编辑帧内的任意区域。在第二阶段，MagicProp将编辑后的帧作为外观参考，并使用自回归渲染方法生成其余帧。为此，我们开发了一种名为PropDPM的基于扩散的条件生成模型，该模型通过参考外观、目标运动及其先前外观作为条件来合成目标帧。自回归编辑方法确保了所得视频的时间一致性。总体而言，MagicProp结合了图像编辑技术的灵活性与自回归建模的优越时间一致性，能够在输入视频的任意区域灵活编辑对象类型和美学风格，同时保持帧间良好的时间一致性。在各种视频编辑场景中的大量实验证明了MagicProp的有效性。
英文摘要：
This paper addresses the issue of modifying the visual appearance of videos while preserving their motion. A novel framework, named MagicProp, is proposed, which disentangles the video editing process into two stages: appearance editing and motion-aware appearance propagation. In the first stage, MagicProp selects a single frame from the input video and applies image-editing techniques to modify the content and/or style of the frame. The flexibility of these techniques enables the editing of arbitrary regions within the frame. In the second stage, MagicProp employs the edited frame as an appearance reference and generates the remaining frames using an autoregressive rendering approach. To achieve this, a diffusion-based conditional generation model, called PropDPM, is developed, which synthesizes the target frame by conditioning on the reference appearance, the target motion, and its previous appearance. The autoregressive editing approach ensures temporal consistency in the resulting videos. Overall, MagicProp combines the flexibility of image-editing techniques with the superior temporal consistency of autoregressive modeling, enabling flexible editing of object types and aesthetic styles in arbitrary regions of input videos while maintaining good temporal consistency across frames. Extensive experiments in various video editing scenarios demonstrate the effectiveness of MagicProp.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
