arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.14645cs.CV
收藏

频谱匹配:一种提升潜在扩散扩散性的统一视角

Spectrum Matching: a Unified Perspective for Superior Diffusability in Latent Diffusion

Mang Ning, Mingxiao Li, Le Zhang, Lanmiao Liu, Matthew B. Blaschko, Albert Ali Salah, Itir Onal Ertugrul

AI总结:

本文研究了变分自编码器在潜在扩散中的扩散性,提出频谱匹配假设,通过频谱匹配提升潜在扩散性,并在多个数据集上验证了其有效性。

摘要:

在本文中,我们研究了变分自编码器(VAE)在潜在扩散中的扩散性(可学习性)。首先,我们显示,使用MSE目标训练的像素空间扩散本质上倾向于学习低和中空间频率,并且自然图像的功率律功率谱密度(PSD)使这种偏差在感知上是有益的。受此结果启发,我们提出了频谱匹配假设:具有优异扩散性的潜在应(i)遵循平坦的功率律PSD(编码频谱匹配,ESM)和(ii)通过解码器保持频率到频率的语义对应(解码频谱匹配,DSM)。在实践中,我们通过匹配图像和潜在的PSD应用ESM,并通过共享频谱掩码和频率对齐的重建应用DSM。重要的是,频谱匹配提供了一种统一的视角,澄清了之前关于过噪或过平潜在的观察,并将几种最近的方法解释为特殊情况(例如,VA-VAE、EQ-VAE)。实验表明,频谱匹配在CelebA和ImageNet数据集上实现了优越的扩散生成,并优于先前的方法。最后,我们将频谱观点扩展到表示对齐(REPA):我们显示目标表示的方向频谱能量对于REPA至关重要,并提出了一种基于DoG的方法来进一步提高REPA的性能。我们的代码可在https://github.com/forever208/SpectrumMatching上获得。

英文摘要:

In this paper, we study the diffusability (learnability) of variational autoencoders (VAEs) in latent diffusion. We first derive distribution-free bounds on the maximum achievable reduction in pixel-space denoising loss. Under a power-law power spectral density (PSD), these bounds establish an ordering between sufficiently separated frequencies, with larger achievable reductions at lower frequencies. Motivated by this analysis and measurements of a trained pixel diffusion model, we seek to make low-frequency denoising gains useful for generating perceptually meaningful semantic content in latent diffusion. We propose the Spectrum Matching Hypothesis: latents with superior diffusability should (i) follow a flattened power-law PSD (Encoding Spectrum Matching, ESM) and (ii) preserve frequency-to-frequency semantic correspondence through the decoder (Decoding Spectrum Matching, DSM). In practice, we apply ESM by matching the PSD between images and latents, and DSM via shared spectral masking with frequency-aligned reconstruction. Importantly, Spectrum Matching clarifies prior observations of over-noisy or over-smoothed latents and provides a unified spectral interpretation of prior methods, including VA-VAE and EQ-VAE. Experiments suggest that Spectrum Matching yields superior diffusion generation on CelebA and ImageNet datasets, and outperforms prior approaches. Finally, we extend the spectral view to representation alignment (REPA): we show that the directional spectral energy of the target representation is crucial for REPA, and propose a DoG-based method to further improve the performance of REPA. Our code is available at https://github.com/forever208/SpectrumMatching.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑