面向鲁棒文本到视频生成的潜变量视频扩散模型腐蚀感知训练
Corruption-Aware Training of Latent Video Diffusion Models for Robust Text-to-Video Generation
AI总结：
本文提出CAT-LVDM框架，通过结构化噪声注入提升视频扩散模型的鲁棒性，实验显示其在多个数据集上优于传统方法，且能扩展至自回归生成和多模态视频理解模型。
摘要：
潜变量视频扩散模型（LVDMs）在图像和视频生成中实现了最先进的生成质量；然而，它们在噪声条件下的鲁棒性较差，文本或多模态嵌入的小扰动会随时间步级联导致语义漂移。现有图像扩散的腐蚀策略（高斯、均匀）在视频设置中失效，因为静态噪声破坏时间一致性。本文提出CAT-LVDM，一种腐蚀感知训练框架，具有针对视频扩散的结构化、数据对齐的噪声注入。我们的两个操作符，批中心噪声注入（BCNI）和频谱感知上下文噪声（SACN），将扰动与批次语义或频谱动态对齐以保持一致性。CAT-LVDM实现了显著提升：BCNI在WebVid-2M、MSR-VTT和MSVD上将FVD减少31.9%，而SACN在UCF-101上提升12.3%，优于高斯、均匀以及大型扩散基线如DEMO（2.3B）和Lavie（3B），尽管训练数据仅为5倍少。消融实验确认了低秩、数据对齐噪声的独特价值，理论证明了这些操作符如何收紧鲁棒性和泛化界限。CAT-LVDM因此设定了新的视频扩散框架，且实验表明它也可扩展至自回归生成和多模态视频理解LLMs。代码、模型和样本可在https://github.com/chikap421/catlvdm获取。
英文摘要：
Latent Video Diffusion Models (LVDMs) have achieved state-of-the-art generative quality for image and video generation; however, they remain brittle under noisy conditioning, where small perturbations in text or multimodal embeddings can cascade over timesteps and cause semantic drift. Existing corruption strategies from image diffusion (Gaussian, Uniform) fail in video settings because static noise disrupts temporal fidelity. In this paper, we propose CAT-LVDM, a corruption-aware training framework with structured, data-aligned noise injection tailored for video diffusion. Our two operators, Batch-Centered Noise Injection (BCNI) and Spectrum-Aware Contextual Noise (SACN), align perturbations with batch semantics or spectral dynamics to preserve coherence. CAT-LVDM yields substantial gains: BCNI reduces FVD by 31.9 percent on WebVid-2M, MSR-VTT, and MSVD, while SACN improves UCF-101 by 12.3 percent, outperforming Gaussian, Uniform, and even large diffusion baselines like DEMO (2.3B) and Lavie (3B) despite training on 5x less data. Ablations confirm the unique value of low-rank, data-aligned noise, and theory establishes why these operators tighten robustness and generalization bounds. CAT-LVDM thus sets a new framework for robust video diffusion, and our experiments show that it can also be extended to autoregressive generation and multimodal video understanding LLMs. Code, models, and samples are available at https://github.com/chikap421/catlvdm
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
