arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.13256cs.AIcs.CVcs.LG
收藏

扩散模型如何学习因式分解与组合

How Diffusion Models Learn to Factorize and Compose

Massachusetts Institute of Technology(麻省理工学院)

Qiyao Liang, Ziming Liu, Mitchell Ostrow, Ila Fiete

AI总结:

本文通过简化的2D高斯凸起实验揭示扩散模型(DDPMs)能学习因式分解表示实现组合泛化,并利用渗流理论解释其机制,发现少量组合样本即可高效训练。

摘要:

扩散模型能够生成结合了训练集中可能未同时出现的元素的逼真图像,展示了组合泛化的能力。尽管如此,组合性的确切机制及其如何通过训练获得仍然难以捉摸。受认知神经科学方法的启发,我们考虑一个高度简化的设定,以检验扩散模型是否以及何时学习可组合特征的语义有意义且因式分解的表示。我们在条件去噪扩散概率模型(DDPMs)上进行了广泛受控实验,训练其生成各种形式的2D高斯凸起图像。我们发现,模型学习了因式分解但非完全连续的流形表示,用于编码数据背后的连续特征变化。借助这些表示,模型展现出卓越的特征组合能力,但在对给定特征未见过的值进行插值方面能力有限。我们的实验结果进一步表明,扩散模型仅需少量组合示例即可实现组合性,这暗示了一种更高效的DDPMs训练方法。最后,我们将扩散模型中的流形形成与物理学中的渗流理论联系起来,为因式分解表示学习的突然出现提供了深刻见解。因此,我们详尽的玩具实验为理解扩散模型如何捕获数据中的组合结构做出了贡献。

英文摘要:

Diffusion models are capable of generating photo-realistic images that combine elements which likely do not appear together in the training set, demonstrating the ability to \textit{compositionally generalize}. Nonetheless, the precise mechanism of compositionality and how it is acquired through training remains elusive. Inspired by cognitive neuroscientific approaches, we consider a highly reduced setting to examine whether and when diffusion models learn semantically meaningful and factorized representations of composable features. We performed extensive controlled experiments on conditional Denoising Diffusion Probabilistic Models (DDPMs) trained to generate various forms of 2D Gaussian bump images. We found that the models learn factorized but not fully continuous manifold representations for encoding continuous features of variation underlying the data. With such representations, models demonstrate superior feature compositionality but limited ability to interpolate over unseen values of a given feature. Our experimental results further demonstrate that diffusion models can attain compositionality with few compositional examples, suggesting a more efficient way to train DDPMs. Finally, we connect manifold formation in diffusion models to percolation theory in physics, offering insight into the sudden onset of factorized representation learning. Our thorough toy experiments thus contribute a deeper understanding of how diffusion models capture compositional structure in data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑