Slicedit：利用时空切片，基于文本到图像扩散模型的零样本视频编辑
Slicedit: Zero-Shot Video Editing With Text-to-Image Diffusion Models Using Spatio-Temporal Slices
AI总结：
针对T2I扩散模型用于视频编辑时难以保持强非刚性运动时间一致性的问题，提出Slicedit，通过同时编辑空间与时空切片实现零样本文本视频编辑。
摘要：
文本到图像（T2I）扩散模型在图像合成与编辑方面取得了最先进的结果。然而，将这类预训练模型用于视频编辑被认为是一项重大挑战。许多现有工作试图通过显式对应机制（无论是在像素空间中还是在深度特征之间）来增强编辑后视频的时间一致性。但这些方法难以处理强烈的非刚性运动。本文提出一种根本不同的方法，其依据是观察到自然视频的时空切片表现出与自然图像相似的特征。因此，通常仅作为视频帧先验使用的同一 T2I 扩散模型，也可通过将其应用于时空切片，作为增强时间一致性的强先验。基于这一观察，我们提出 Slicedit，这是一种基于文本的视频编辑方法，利用预训练 T2I 扩散模型同时处理空间切片和时空切片。我们的方法生成的视频在遵循目标文本的同时，保留原始视频的结构和运动。通过大量实验，我们证明了 Slicedit 编辑各种真实世界视频的能力，证实了其相较于现有竞争方法的明显优势。网页：https://matankleiner.github.io/slicedit/
英文摘要：
Text-to-image (T2I) diffusion models achieve state-of-the-art results in image synthesis and editing. However, leveraging such pretrained models for video editing is considered a major challenge. Many existing works attempt to enforce temporal consistency in the edited video through explicit correspondence mechanisms, either in pixel space or between deep features. These methods, however, struggle with strong nonrigid motion. In this paper, we introduce a fundamentally different approach, which is based on the observation that spatiotemporal slices of natural videos exhibit similar characteristics to natural images. Thus, the same T2I diffusion model that is normally used only as a prior on video frames, can also serve as a strong prior for enhancing temporal consistency by applying it on spatiotemporal slices. Based on this observation, we present Slicedit, a method for text-based video editing that utilizes a pretrained T2I diffusion model to process both spatial and spatiotemporal slices. Our method generates videos that retain the structure and motion of the original video while adhering to the target text. Through extensive experiments, we demonstrate Slicedit's ability to edit a wide range of real-world videos, confirming its clear advantages compared to existing competing methods. Webpage: https://matankleiner.github.io/slicedit/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
