arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.21085cs.CV
收藏

通过方差扩展损失镇定采样扰动以提升潜在扩散模型

Taming Sampling Perturbations with Variance Expansion Loss for Latent Diffusion Models

University of Electronic Science and Technology of China(电子科学与技术大学)

Qifan Li, Xingyu Zhou, Jinhua Zhang, Weiyi You, Shuhang Gu

AI总结:

本文提出方差扩展损失,通过对抗重建与方差扩展的交互,提升潜在空间鲁棒性,改进扩散生成质量。

摘要:

潜在扩散模型因其在紧凑潜在空间中学习扩散过程的能力,已成为高保真和高效图像生成的主导框架。然而,尽管先前研究主要关注潜在空间的重建准确性和语义对齐,我们发现另一个关键因素,即对采样扰动的鲁棒性,在决定生成质量中也起着重要作用。通过实证和理论分析,我们表明潜在扩散模型中常用的β-VAE基于的分词器倾向于生成过于紧凑的潜在流形,这些流形在扩散采样过程中对随机扰动高度敏感,导致视觉退化。为了解决这个问题,我们提出了一种简单而有效的解决方案,通过引入方差扩展损失,构建一个对采样扰动鲁棒的潜在空间,同时保持强大的重建保真度。该方法通过对抗重建与方差扩展的交互,实现一种自适应平衡,从而在保持重建准确性的同时提高对随机采样扰动的鲁棒性。大量实验表明,我们的方法在不同潜在扩散架构中一致提高了生成质量,证实了潜在空间的鲁棒性是实现稳定和忠实扩散采样的关键缺失要素。

英文摘要:

Latent diffusion models have emerged as the dominant framework for high-fidelity and efficient image generation, owing to their ability to learn diffusion processes in compact latent spaces. However, while previous research has focused primarily on reconstruction accuracy and semantic alignment of the latent space, we observe that another critical factor, robustness to sampling perturbations, also plays a crucial role in determining generation quality. Through empirical and theoretical analyses, we show that the commonly used $β$-VAE-based tokenizers in latent diffusion models, tend to produce overly compact latent manifolds that are highly sensitive to stochastic perturbations during diffusion sampling, leading to visual degradation. To address this issue, we propose a simple yet effective solution that constructs a latent space robust to sampling perturbations while maintaining strong reconstruction fidelity. This is achieved by introducing a Variance Expansion loss that counteracts variance collapse and leverages the adversarial interplay between reconstruction and variance expansion to achieve an adaptive balance that preserves reconstruction accuracy while improving robustness to stochastic sampling. Extensive experiments demonstrate that our approach consistently enhances generation quality across different latent diffusion architectures, confirming that robustness in latent space is a key missing ingredient for stable and faithful diffusion sampling.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑