arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2605.14136cs.CV
收藏

TeDiO:基于时间对角优化的训练自由一致视频扩散

TeDiO: Temporal Diagonal Optimization for Training-Free Coherent Video Diffusion

UNC Chapel Hill(北卡罗来纳大学教堂山分校);Futurewei Technologies Inc(未来科技有限公司);UCSD(加州大学圣迭戈分校)

Nurislam Tursynbek, Zhiqiang Lao, Heather Yu, Gedas Bertasius, Marc Niethammer

AI总结:

TeDiO通过优化内部注意力模式提升视频生成的时序一致性,无需训练或外部监督,实现更流畅的动态表现。

摘要:

近期文本到视频扩散变换器能生成视觉吸引人的帧,但仍在时序一致性上面临挑战,常产生闪烁、漂移或不稳定运动。我们发现这些失败在模型内部留下明显印记:不一致的视频在中间自注意力图中表现出不规则、碎片化的对角线模式,而稳定的运动对应平滑的带状对角线模式。基于此观察,我们引入TeDiO,一种无需训练、在推理时使用的 方法,通过正则化这些内部注意力模式来增强时序一致性。TeDiO估计对角线平滑度,识别不稳定区域,并执行轻量级的潜在更新,促进一致的帧间动态,不修改模型权重或使用外部运动监督。在多个视频扩散模型(如Wan2.1、CogVideoX)上,TeDiO实现了明显更平滑的运动,同时保持每帧视觉质量,提供了一种高效的即插即用方法,以提升现代视频生成系统中的动态真实感。

英文摘要:

Recent text-to-video diffusion transformers generate visually compelling frames, yet still struggle with temporal coherence, often producing flickering, drifting, or unstable motion. We show that these failures leave a clear imprint inside the model: incoherent videos consistently exhibit irregular, fragmented temporal diagonals in their intermediate self-attention maps, whereas stable motion corresponds to smooth, band-diagonal patterns. Building on this observation, we introduce TeDiO, a training-free, inference-time method that reinforces temporal consistency by regularizing these internal attention patterns. TeDiO estimates diagonal smoothness, identifies unstable regions, and performs lightweight latent updates that promote coherent frame-to-frame dynamics, without modifying model weights or using external motion supervision. Across multiple video diffusion models (e.g., Wan2.1, CogVideoX), TeDiO delivers markedly smoother motion while preserving per-frame visual quality, offering an efficient plug-and-play approach to improving dynamic realism in modern video generation systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑