arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.09358cs.LG
收藏

通过模式插值理解扩散模型中的幻觉

Understanding Hallucinations in Diffusion Models through Mode Interpolation

Carnegie Mellon University(卡内基梅隆大学);DatologyAI

Sumukh K Aithal, Pratyush Maini, Zachary C. Lipton, J. Zico Kolter

AI总结:

本文研究了扩散模型中由模式插值导致的幻觉失效模式,发现解码器不连续的损失景观会引发此现象,并提出基于反向采样轨迹方差的指标,在移除超95%幻觉的同时保留96%支持域内样本,证明了其对稳定递归训练的作用。

摘要:

通俗地讲,基于扩散过程的图像生成模型经常被说成会表现出“幻觉”,即生成训练数据中永远不会出现的样本。但这种幻觉从何而来?在本文中,我们研究了扩散模型中的一种特定失效模式,我们将其称为模式插值(mode interpolation)。具体而言,我们发现扩散模型会在训练集中的相邻数据模式之间平滑地“插值”,以生成完全处于原始训练分布支持域之外的样本;这种现象导致扩散模型生成真实数据中从未存在过的伪影(即幻觉)。我们系统地研究了该现象产生的原因及其表现形式。通过对一维和二维高斯分布的实验,我们展示了扩散模型解码器中不连续的损失景观如何导致一个区域,在该区域中任何平滑近似都会引发此类幻觉。通过对具有各种形状的人工数据集的实验,我们展示了幻觉如何导致生成从未存在过的形状组合。最后,我们表明扩散模型实际上知道它们何时超出支持域并产生幻觉。这体现在生成样本在最后几次反向采样过程中的轨迹具有高方差。使用一个简单的指标来捕捉这种方差,我们可以在生成时移除超过95%的幻觉,同时保留96%的支持域内样本。我们通过在MNIST和二维高斯数据集上的实验,展示了此类幻觉(及其移除)对合成数据递归训练崩溃(及稳定)的影响,以此结束我们的探索。我们在 https://github.com/locuslab/diffusion-model-hallucination 发布了代码。

英文摘要:

Colloquially speaking, image generation models based upon diffusion processes are frequently said to exhibit "hallucinations," samples that could never occur in the training data. But where do such hallucinations come from? In this paper, we study a particular failure mode in diffusion models, which we term mode interpolation. Specifically, we find that diffusion models smoothly "interpolate" between nearby data modes in the training set, to generate samples that are completely outside the support of the original training distribution; this phenomenon leads diffusion models to generate artifacts that never existed in real data (i.e., hallucinations). We systematically study the reasons for, and the manifestation of this phenomenon. Through experiments on 1D and 2D Gaussians, we show how a discontinuous loss landscape in the diffusion model's decoder leads to a region where any smooth approximation will cause such hallucinations. Through experiments on artificial datasets with various shapes, we show how hallucination leads to the generation of combinations of shapes that never existed. Finally, we show that diffusion models in fact know when they go out of support and hallucinate. This is captured by the high variance in the trajectory of the generated sample towards the final few backward sampling process. Using a simple metric to capture this variance, we can remove over 95% of hallucinations at generation time while retaining 96% of in-support samples. We conclude our exploration by showing the implications of such hallucination (and its removal) on the collapse (and stabilization) of recursive training on synthetic data with experiments on MNIST and 2D Gaussians dataset. We release our code at https://github.com/locuslab/diffusion-model-hallucination.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑