扩散模型中可重复性与泛化性的出现
The Emergence of Reproducibility and Generalizability in Diffusion Models
AI总结:
研究发现扩散模型在相同初始噪声和确定性采样器下,不同模型输出高度相似,且这种可重复性在记忆和泛化两种训练模式下均存在,对训练效率、模型隐私等有重要启示。
摘要:
在这项工作中,我们研究了扩散模型的一个有趣且普遍的现象,我们称之为“一致模型可重复性”:给定相同的起始噪声输入和确定性采样器,不同的扩散模型通常会产生非常相似的输出。我们通过全面的实验证实了这一现象,这意味着不同的扩散模型一致地达到相同的数据分布和评分函数,无论扩散模型框架、模型架构或训练过程如何。更引人注目的是,我们的进一步研究表明,扩散模型学习到的不同分布受到训练数据大小的影响。这一点得到了以下事实的支持:模型可重复性表现在两种不同的训练机制中:(i)“记忆机制”,其中扩散模型过拟合到训练数据分布,以及(ii)“泛化机制”,其中模型学习底层数据分布。我们的研究还发现,这一有价值的特性推广到许多扩散模型的变体,包括用于条件使用、解决逆问题和模型微调的变体。最后,我们的工作提出了许多有趣的理论问题供未来研究,并强调了关于训练效率、模型隐私和扩散模型受控生成的实际意义。
英文摘要:
In this work, we investigate an intriguing and prevalent phenomenon of diffusion models which we term as "consistent model reproducibility": given the same starting noise input and a deterministic sampler, different diffusion models often yield remarkably similar outputs. We confirm this phenomenon through comprehensive experiments, implying that different diffusion models consistently reach the same data distribution and scoring function regardless of diffusion model frameworks, model architectures, or training procedures. More strikingly, our further investigation implies that diffusion models are learning distinct distributions affected by the training data size. This is supported by the fact that the model reproducibility manifests in two distinct training regimes: (i) "memorization regime", where the diffusion model overfits to the training data distribution, and (ii) "generalization regime", where the model learns the underlying data distribution. Our study also finds that this valuable property generalizes to many variants of diffusion models, including those for conditional use, solving inverse problems, and model fine-tuning. Finally, our work raises numerous intriguing theoretical questions for future investigation and highlights practical implications regarding training efficiency, model privacy, and the controlled generation of diffusion models.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。