arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新，休息一下吧，周末愉快~~
arXiv 2501.09019cs.CV
收藏

Ouroboros-Diffusion：探索免调优长视频扩散中的一致性内容生成

Ouroboros-Diffusion: Exploring Consistent Content Generation in Tuning-free Long Video Diffusion

Jingyuan Chen, Fuchen Long, Jie An, Zhaofan Qiu, Ting Yao, Jiebo Luo, Tao Mei

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对免调优长视频扩散的长程一致性难题，提出Ouroboros-Diffusion框架，通过尾部潜在采样、主体感知跨帧注意力和自循环引导提升一致性，在VBench基准上表现优于现有方法。

摘要：

基于预训练文本到视频模型构建的先进先出（FIFO）视频扩散，近期成为免调优长视频生成的有效方法。该技术维护一个噪声逐步递增的视频帧队列，在队列头部持续生成干净帧，同时在尾部将高斯噪声入队。但FIFO-Diffusion因缺乏帧间对应关系建模，往往难以在生成视频中保持长程时序一致性。本文提出Ouroboros-Diffusion，这是一种新型视频去噪框架，旨在增强结构与内容（主体）一致性，可生成任意长度的一致性视频。具体而言，我们在队列尾部引入一种新的潜在采样技术以提升结构一致性，确保帧间感知上的平滑过渡。为增强主体一致性，我们设计了主体感知跨帧注意力（SACFA）机制，在短片段内对齐跨帧主体以实现更好的视觉连贯性。此外，我们引入自循环引导，该技术利用队列前端所有先前的更干净帧的信息，引导尾部噪声更大的帧的去噪过程，促进丰富且具上下文的全局信息交互。在VBench基准上开展的大量长视频生成实验证明了Ouroboros-Diffusion的优越性，尤其在主体一致性、运动平滑度及时序一致性方面表现突出。

英文摘要：

The first-in-first-out (FIFO) video diffusion, built on a pre-trained text-to-video model, has recently emerged as an effective approach for tuning-free long video generation. This technique maintains a queue of video frames with progressively increasing noise, continuously producing clean frames at the queue's head while Gaussian noise is enqueued at the tail. However, FIFO-Diffusion often struggles to keep long-range temporal consistency in the generated videos due to the lack of correspondence modeling across frames. In this paper, we propose Ouroboros-Diffusion, a novel video denoising framework designed to enhance structural and content (subject) consistency, enabling the generation of consistent videos of arbitrary length. Specifically, we introduce a new latent sampling technique at the queue tail to improve structural consistency, ensuring perceptually smooth transitions among frames. To enhance subject consistency, we devise a Subject-Aware Cross-Frame Attention (SACFA) mechanism, which aligns subjects across frames within short segments to achieve better visual coherence. Furthermore, we introduce self-recurrent guidance. This technique leverages information from all previous cleaner frames at the front of the queue to guide the denoising of noisier frames at the end, fostering rich and contextual global information interaction. Extensive experiments of long video generation on the VBench benchmark demonstrate the superiority of our Ouroboros-Diffusion, particularly in terms of subject consistency, motion smoothness, and temporal consistency.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑