arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2206.06227cs.LGmath.PRmath.STstat.MLstat.TH
收藏

具有多项式复杂度的基于分数的生成建模的收敛性

Convergence for score-based generative modeling with polynomial complexity

Johns Hopkins University(约翰斯·霍普金斯大学);Duke University(杜克大学)

Holden Lee, Jianfeng Lu, Yixin Tan

AI总结:

该研究证明了基于分数的生成建模核心机制的首个多项式收敛保证,规避了指数误差与维数灾难,为退火过程的必要性提供理论支撑,且验证了预测-校正算法的更优收敛性。

摘要:

基于分数的生成建模(SGM)是一种从数据中学习概率分布并生成更多样本的极为成功的方法。我们证明了SGM核心机制的首个多项式收敛保证:在给定L²(p)精度的分数估计(即∇ln p的估计)的情况下,从概率密度p中采样。与以往研究相比,我们的方法不会产生随时间指数增长的误差,也不会遭受维数灾难的影响。我们的保证适用于任何光滑分布,且收敛性与其对数Sobolev常数呈多项式依赖关系。利用这一保证,我们对基于分数的生成建模进行了理论分析,该模型在给定不同噪声尺度下的分数估计时,可将白噪声输入转换为来自已学习数据分布的样本。我们的分析为实践中需要退火过程才能生成高质量样本的观察结果提供了理论依据,因为我们的证明本质上依赖于使用退火来在每一步获得热启动。此外,我们还证明了预测-校正算法比单独使用其中任一部分都能获得更好的收敛效果。

英文摘要:

Score-based generative modeling (SGM) is a highly successful approach for learning a probability distribution from data and generating further samples. We prove the first polynomial convergence guarantees for the core mechanic behind SGM: drawing samples from a probability density $p$ given a score estimate (an estimate of $\nabla \ln p$) that is accurate in $L^2(p)$. Compared to previous works, we do not incur error that grows exponentially in time or that suffers from a curse of dimensionality. Our guarantee works for any smooth distribution and depends polynomially on its log-Sobolev constant. Using our guarantee, we give a theoretical analysis of score-based generative modeling, which transforms white-noise input into samples from a learned data distribution given score estimates at different noise scales. Our analysis gives theoretical grounding to the observation that an annealed procedure is required in practice to generate good samples, as our proof depends essentially on using annealing to obtain a warm start at each step. Moreover, we show that a predictor-corrector algorithm gives better convergence than using either portion alone.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑