arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2209.11215cs.LGmath.STstat.TH
收藏

采样与学习得分一样简单:最小数据假设下扩散模型的理论

Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions

University of California, Berkeley(加州大学伯克利分校);Massachusetts Institute of Technology(麻省理工学院);Microsoft Research(微软研究院);Carnegie Mellon University(卡内基梅隆大学);Duke University(杜克大学)

Sitan Chen, Sinho Chewi, Jerry Li, Yuanzhi Li, Adil Salim, Anru R. Zhang

AI总结:

本文在仅需L2准确得分估计和最小数据假设下,为基于得分的生成模型建立了高效采样理论,证明其复杂度与朗之万扩散离散化的最优保证相当,并指出临界阻尼朗之万扩散并不能降低模型复杂度。

摘要:

我们为基于得分的生成模型(SGMs),例如去噪扩散概率模型(DDPMs),提供了理论收敛保证。这类模型构成了大规模现实世界生成模型(如DALL·E 2)的骨干。我们的主要结果是,在假设得分估计准确的前提下,此类SGMs能够从本质上任意现实的数据分布中高效采样。与先前工作相比,我们的结果具有以下特点:(1)适用于L2准确的得分估计(而非L∞准确的);(2)不要求排除显著非对数凹性的限制性函数不等式条件;(3)在所有相关问题参数上具有多项式规模;(4)在得分误差足够小的情况下,与朗之万扩散离散化的最先进复杂度保证相匹配。我们认为这是对SGMs经验成功的有力理论论证。我们还研究了基于临界阻尼朗之万扩散(CLD)的SGMs。与传统观点相反,我们提供了证据表明使用CLD并不会降低SGMs的复杂度。

英文摘要:

We provide theoretical convergence guarantees for score-based generative models (SGMs) such as denoising diffusion probabilistic models (DDPMs), which constitute the backbone of large-scale real-world generative models such as DALL$\cdot$E 2. Our main result is that, assuming accurate score estimates, such SGMs can efficiently sample from essentially any realistic data distribution. In contrast to prior works, our results (1) hold for an $L^2$-accurate score estimate (rather than $L^\infty$-accurate); (2) do not require restrictive functional inequality conditions that preclude substantial non-log-concavity; (3) scale polynomially in all relevant problem parameters; and (4) match state-of-the-art complexity guarantees for discretization of the Langevin diffusion, provided that the score error is sufficiently small. We view this as strong theoretical justification for the empirical success of SGMs. We also examine SGMs based on the critically damped Langevin diffusion (CLD). Contrary to conventional wisdom, we provide evidence that the use of the CLD does not reduce the complexity of SGMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑