arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.13637cs.CV

DCDM:分而治之扩散模型用于保持一致性视频生成

DCDM: Divide-and-Conquer Diffusion Models for Consistency-Preserving Video Generation

Fudan University(复旦大学);Tencent(腾讯)

Haoyu Zhao, Yuang Zhang, Junqi Cheng, Jiaxi Gu, Zenghui Lu, Peng Shu, Zuxuan Wu, Yu-Gang Jiang

AI总结:

DCDM通过分而治之的扩散模型,解决视频生成中的语义、几何和身份一致性问题,提升视频生成的连贯性和可控性。

摘要:

最近的视频生成模型在视觉保真度方面表现出色,但往往在语义、几何和身份一致性方面存在困难。在本文中,我们提出了一种系统级框架,称为分而治之扩散模型(DCDM),以解决三个关键挑战:(1)片段内世界知识一致性,(2)片段间摄像机一致性,(3)片段间元素一致性。DCDM将这些场景下的视频一致性建模分解为三个专用组件,同时共享统一的视频生成主干。对于片段内一致性,DCDM利用大型语言模型将输入提示解析为结构化的语义表示,随后由扩散变压器翻译成连贯的视频内容。对于片段间摄像机一致性,我们提出了一种在噪声空间中的时间摄像机表示,能够实现精确且稳定的摄像机运动控制,以及一种文本到图像初始化机制,以进一步增强可控性。对于片段间一致性,DCDM采用了一种整体场景生成范式,结合了窗口交叉注意力和稀疏片段间自注意力,确保长时间叙述的一致性,同时保持计算效率。我们在AAAI'26 CVM竞赛的测试集上验证了我们的框架,结果表明所提出的方法有效解决了这些挑战。

英文摘要:

Recent video generative models have demonstrated impressive visual fidelity, yet they often struggle with semantic, geometric, and identity consistency. In this paper, we propose a system-level framework, termed the Divide-and-Conquer Diffusion Model (DCDM), to address three key challenges: (1) intra-clip world knowledge consistency, (2) inter-clip camera consistency, and (3) inter-shot element consistency. DCDM decomposes video consistency modeling under these scenarios into three dedicated components while sharing a unified video generation backbone. For intra-clip consistency, DCDM leverages a large language model to parse input prompts into structured semantic representations, which are subsequently translated into coherent video content by a diffusion transformer. For inter-clip camera consistency, we propose a temporal camera representation in the noise space that enables precise and stable camera motion control, along with a text-to-image initialization mechanism to further enhance controllability. For inter-shot consistency, DCDM adopts a holistic scene generation paradigm with windowed cross-attention and sparse inter-shot self-attention, ensuring long-range narrative coherence while maintaining computational efficiency. We validate our framework on the test set of the CVM Competition at AAAI'26, and the results demonstrate that the proposed strategies effectively address these challenges.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑