子流形假设下扩散模型奇异性的数学分析
Mathematical analysis of singularities in the diffusion model under the submanifold assumption
AI总结:
针对集中于低维流形的奇异数据分布,本文分析发现DDPM平均漂移函数和SGM得分函数在采样末期渐近爆炸,故提出新的有界目标函数及损失以解决该问题,并通过数值示例验证理论。
摘要:
本文关注机器学习中扩散模型的数学分析。反向采样过程的漂移项表示为涉及数据分布和正向扩散的条件期望。训练过程旨在通过最小化与条件期望相关的均方残差来找到此类漂移函数。利用正向扩散格林函数的小时间近似,我们证明,对于集中在低维流形上的奇异数据分布,DDPM中的解析平均漂移函数和SGM中的得分函数在采样过程的最后阶段渐近爆炸,因此难以通过网络近似。为克服这一困难,我们推导了一个新的目标函数及相关损失,即使对于奇异数据分布,该函数仍保持有界。我们通过多个数值示例验证了理论发现。
英文摘要:
This paper concerns the mathematical analyses of the diffusion model in machine learning. The drift term of the backward sampling process is represented as a conditional expectation involving the data distribution and the forward diffusion. The training process aims to find such a drift function by minimizing the mean-squared residue related to the conditional expectation. Using small-time approximations of the Green's function of the forward diffusion, we show that the analytical mean drift function in DDPM and the score function in SGM asymptotically blow up in the final stages of the sampling process for singular data distributions such as those concentrated on lower-dimensional manifolds, and are therefore difficult to approximate by a network. To overcome this difficulty, we derive a new target function and associated loss, which remains bounded even for singular data distributions. We validate the theoretical findings with several numerical examples.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。