arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.08780cs.CV

超越一致性:在零样本视频编辑中保留时间结构

Beyond Consistency: Preserving Temporal Structure in Zero-Shot Video Editing

Anhui University(安徽大学);University of Surrey(萨里大学);University of Warwick(华威大学)

Deyin Liu, Yisheng Ding, Zhe Jin, Xiatian Zhu, Anjan Dutta, Lin Wu

AI总结:

提出一种零样本视频编辑方法,通过自适应分割视频片段、选取锚帧和令牌合并策略,首次显式保留源视频的时间结构,平衡编辑保真度与计算效率。

摘要:

现有的零样本视频编辑方法依赖预训练的扩散模型,成功实现了空间控制和基本的时间一致性,但根本上未能保留视频的原始时间结构。这一区别至关重要:时间一致性确保视觉平滑,而时间结构决定了视频的高层叙事、节奏和语义流。没有这种保留,编辑输出(尤其是具有复杂语义变化的长视频)在叙事上变得不连贯,语义模糊。为了解决这一局限性,我们提出了一种新颖的零样本编辑方法,首次明确关注保留源视频的时间结构。我们通过基于特征相似性自适应地将视频分割成语义不同的片段,并为每个片段选择一个代表性的锚帧来实现这一点。为了增强片段内保真度和计算效率,我们设计了一种片段自适应的令牌合并策略,利用锚帧的语义主导性来稳定编辑。此外,我们采用交替组合策略,确保片段间无缝过渡,同时保持语义区分。大量实验表明,我们的方法达到了最先进的结果,成功平衡了原始时间结构的保留与计算效率,为零样本视频编辑保真度设立了新基准。

英文摘要:

Existing zero-shot video editing methods rely on pre-trained diffusion models, successfully achieving spatial control and basic temporal consistency but fundamentally fail to preserve the video's original temporal structure.This distinction is critical: temporal consistency ensures visual smoothness, but temporal structure dictates the video's high-level narrative, rhythm, and semantic flow. Without this preservation, the edited output, especially for long videos with complex semantic variations, becomes narratively incoherent and semantically ambiguous. To address this limitation, we introduce a novel zero-shot editing approach that, for the first time, explicitly focuses on preserving the source video's temporal structure. We achieve this by adaptively partitioning the video into semantically distinct clips based on feature similarity and selecting a representative anchor frame for each clip. To enhance both intra-clip fidelity and computational efficiency, we design a clip-adaptive token merging strategy which leverages the anchor's semantic dominance to stabilize the editing. Furthermore, we employ an alternating combination strategy that ensures seamless inter-clip transitions while maintaining semantic distinction. Extensive experiments demonstrate that our method achieves state-of-the-art results, successfully balancing the preservation of original temporal structure with computational efficiency, and setting a new benchmark for zero-shot video editing fidelity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑