arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.11319cs.LGstat.ML
收藏

关于对数动力学对分数函数误差的鲁棒性

On the Robustness of Langevin Dynamics to Score Function Error

Cornell University(康奈尔大学)

Daniel Yiming Cao, August Y. Chen, Karthik Sridharan, Yuchen Wu

AI总结:

本文研究了基于分数函数的生成模型对分数函数估计误差的鲁棒性,发现对数动力学在L2误差(更一般地Lp误差)下并不鲁棒,即使在高维简单分布中,即使分数函数估计误差非常小,对数动力学在多项式时间内运行也会导致与目标分布的总变差距离很大,这进一步支持了扩散模型优于对数动力学。

摘要:

我们考虑了基于分数函数的生成模型对分数函数估计误差的鲁棒性。特别是,我们证明了对数动力学对分数函数估计的L2误差(更一般地Lp误差)不具有鲁棒性。已知在分数函数估计的L2误差较小的情况下,扩散模型可以在多项式时间内忠实采样目标分布,只要满足一定的正则性假设。相比之下,我们的工作表明,即使对于高维简单分布,对数动力学在任何多项式时间内运行都会产生与目标分布在总变差(TV)距离远的分布,即使分数函数估计的L2误差(更一般地Lp误差)可以任意小。考虑到在实践中从数据学习分数函数时,分数函数估计误差是不可避免的,我们的结果进一步支持扩散模型优于对数动力学,并警示不要使用估计的分数函数进行对数动力学采样。

英文摘要:

We consider the robustness of score-based generative modeling to errors in the estimate of the score function. In particular, we show that Langevin dynamics is not robust to the $L^2$ errors (more generally $L^p$ errors) in the estimate of the score function. It is well-established that with small $L^2$ errors in the estimate of the score function, diffusion models can sample faithfully from the target distribution under fairly mild regularity assumptions in a polynomial time horizon. In contrast, our work shows that even for simple distributions in high dimensions, Langevin dynamics run for any polynomial time horizon will produce a distribution far from the target distribution in Total Variation (TV) distance, even when the $L^2$ error (more generally $L^p$) of the estimate of the score function is arbitrarily small. Considering such an error in the estimate of the score function is unavoidable in practice when learning the score function from data, our results provide further justification for diffusion models over Langevin dynamics and serve to caution against the use of Langevin dynamics with estimated scores.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑