Ground-A-Video：利用文本到图像扩散模型实现零样本定位视频编辑
Ground-A-Video: Zero-shot Grounded Video Editing using Text-to-image Diffusion Models
AI总结：
本文提出Ground-A-Video框架，通过跨帧门控注意力、调制交叉注意力和光流引导平滑，以零样本方式实现时间一致的多属性视频编辑，在编辑准确性和帧一致性上优于基线方法。
摘要：
近年来，视频编辑领域的探索在单属性编辑或风格迁移任务中展现了令人鼓舞的成果，其途径要么是在文本-视频数据上训练文本到视频（T2V）模型，要么是采用免训练方法。然而，当面对多属性编辑场景的复杂性时，这些方法暴露出诸多不足，例如遗漏或忽略预期的属性变化、修改输入视频中的错误元素，以及未能保留输入视频中本应保持不变的区域。为解决这一问题，我们在此提出一种新颖的定位引导的视频到视频转换框架，称为Ground-A-Video，用于多属性视频编辑。Ground-A-Video以免训练的方式实现输入视频在时间上一致的多属性编辑，同时避免了上述不足。我们方法的核心是引入了跨帧门控注意力（Cross-Frame Gated Attention），它以时间一致的方式将定位信息融入潜在表示中，同时结合了调制交叉注意力（Modulated Cross-Attention）和光流引导的反转潜在变量平滑。大量实验和应用表明，Ground-A-Video的零样本能力在编辑准确性和帧一致性方面优于其他基线方法。更多结果和代码可在我们的项目页面（http://ground-a-video.github.io）获取。
英文摘要：
Recent endeavors in video editing have showcased promising results in single-attribute editing or style transfer tasks, either by training text-to-video (T2V) models on text-video data or adopting training-free methods. However, when confronted with the complexities of multi-attribute editing scenarios, they exhibit shortcomings such as omitting or overlooking intended attribute changes, modifying the wrong elements of the input video, and failing to preserve regions of the input video that should remain intact. To address this, here we present a novel grounding-guided video-to-video translation framework called Ground-A-Video for multi-attribute video editing. Ground-A-Video attains temporally consistent multi-attribute editing of input videos in a training-free manner without aforementioned shortcomings. Central to our method is the introduction of Cross-Frame Gated Attention which incorporates groundings information into the latent representations in a temporally consistent fashion, along with Modulated Cross-Attention and optical flow guided inverted latents smoothing. Extensive experiments and applications demonstrate that Ground-A-Video's zero-shot capacity outperforms other baseline methods in terms of edit-accuracy and frame consistency. Further results and code are available at our project page (http://ground-a-video.github.io).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
