一段视频抵得上256个基：面向零样本视频编辑的时空期望最大化反演
A Video is Worth 256 Bases: Spatial-Temporal Expectation-Maximization Inversion for Zero-Shot Video Editing
AI总结：
针对现有视频编辑反演方法时间一致性不足的问题，提出时空期望最大化（STEM）反演方法，通过低秩全局基表示视频，在两种SOTA视频编辑方法上实现了一致的性能提升。
摘要：
本文提出了一种用于零样本视频编辑的视频反演方法，该方法在反演过程中用低秩表示对输入视频建模。现有视频编辑方法通常在编辑前采用典型的2D DDIM反演或朴素的时空DDIM反演，利用每帧的时变表示推导含噪潜变量。与大多数现有方法不同，我们提出时空期望最大化（STEM）反演，该方法以期望最大化方式构建密集视频特征，并迭代估计一组更紧凑的基来表示整段视频。每帧均采用固定的全局表示进行反演，更有利于重建和编辑过程中的时间一致性。大量定性和定量实验表明，我们的STEM反演在两种最先进的视频编辑方法上均能实现一致的性能提升。项目页面：https://stem-inv.github.io/page/。
英文摘要：
This paper presents a video inversion approach for zero-shot video editing, which models the input video with low-rank representation during the inversion process. The existing video editing methods usually apply the typical 2D DDIM inversion or naive spatial-temporal DDIM inversion before editing, which leverages time-varying representation for each frame to derive noisy latent. Unlike most existing approaches, we propose a Spatial-Temporal Expectation-Maximization (STEM) inversion, which formulates the dense video feature under an expectation-maximization manner and iteratively estimates a more compact basis set to represent the whole video. Each frame applies the fixed and global representation for inversion, which is more friendly for temporal consistency during reconstruction and editing. Extensive qualitative and quantitative experiments demonstrate that our STEM inversion can achieve consistent improvement on two state-of-the-art video editing methods. Project page: https://stem-inv.github.io/page/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
