STR-Match：面向无训练视频编辑的时空相关性得分匹配
STR-Match: Matching SpatioTemporal Relevance Score for Training-Free Video Editing
AI总结：
针对视频编辑中时空像素相关性建模不足导致的时序不一致等问题，提出无训练算法STR-Match，利用T2V扩散模型的2D空间注意力和1D时间模块计算STR得分引导潜优化，在重大领域变换下仍生成时序一致且视觉忠实的视频。
摘要：
先前的文本引导视频编辑方法常受时间不一致、运动失真及最显著的领域变换受限等问题困扰。我们将这些局限性归因于编辑过程中对时空像素相关性建模不足。为解决此问题，我们提出STR-Match，一种无训练视频编辑算法，通过由我们新颖的STR得分引导的潜空间优化，生成视觉吸引力强且时空连贯的视频。该得分利用文本到视频（T2V）扩散模型中的2D空间注意力和1D时间模块，捕捉相邻帧间的时空像素相关性，无需计算开销昂贵的3D注意力机制。结合潜掩码集成至潜优化框架中，STR-Match生成时间一致且视觉忠实的视频，在重大领域变换下仍保持强劲性能，同时保留源视频的关键视觉属性。大量实验表明，STR-Match在视觉质量和时空一致性上均持续超越现有方法。
英文摘要：
Previous text-guided video editing methods often suffer from temporal inconsistency, motion distortion, and-most notably-limited domain transformation. We attribute these limitations to insufficient modeling of spatiotemporal pixel relevance during the editing process. To address this, we propose STR-Match, a training-free video editing algorithm that produces visually appealing and spatiotemporally coherent videos through latent optimization guided by our novel STR score. The score captures spatiotemporal pixel relevance across adjacent frames by leveraging 2D spatial attention and 1D temporal modules in text-to-video (T2V) diffusion models, without the overhead of computationally expensive 3D attention mechanisms. Integrated into a latent optimization framework with a latent mask, STR-Match generates temporally consistent and visually faithful videos, maintaining strong performance even under significant domain transformations while preserving key visual attributes of the source. Extensive experiments demonstrate that STR-Match consistently outperforms existing methods in both visual quality and spatiotemporal consistency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
