arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.07514cs.LGcs.AIcs.CV
收藏

基于分数的和漂移模型的统一视角

A Unified View of Score-Based and Drifting Models

Sony AI(索尼人工智能);Sony Group Corporation(索尼集团);Stanford University(斯坦福大学);Georgia Tech(佐治亚理工学院)

Chieh-Hsin Lai, Bac Nguyen, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yuki Mitsufuji, Stefano Ermon, Molei Tao

AI总结:

本文揭示漂移模型与基于分数生成模型的紧密联系,证明漂移本质上是分数匹配目标,通过核估计非参数化实现,而扩散模型则参数化学习。

摘要:

漂移模型通过优化由核诱导的均移差异来训练一步生成器,实践中默认使用拉普拉斯核。在每个点,该差异比较核加权向附近数据样本的位移与向附近模型样本的位移,从而定义生成样本的运输方向。本文证明漂移与基于分数的生成模型更为紧密相关,建立了与扩散模型底层分数匹配原则的精确联系。对于高斯核,总体均移场恰好等于高斯平滑数据和模型分布的分数差(即梯度-密度差)。这一恒等式源于 Tweedie 公式,该公式将高斯平滑密度的分数与其条件均值联系起来,并意味着高斯核漂移是平滑分布上的精确分数匹配目标。更一般地,我们推导了径向核的精确分解,其中均移等于基于分数的场加残差项。对于实用的拉普拉斯核,我们进一步理论和实证证明在高维情况下残差可忽略,意味着实际使用的运输场几乎基于分数。我们的结果揭示了与扩散模型的结构联系:两种方法均使用分数不匹配的运输方向,但漂移通过核估计非参数化实现,而扩散模型则通过神经网络参数化学习。

英文摘要:

Drifting models train one-step generators by optimizing a kernel-induced mean-shift discrepancy between the data and model distributions, with Laplace kernels used by default in practice. At each point, this discrepancy compares the kernel-weighted displacement toward nearby data samples with the corresponding displacement toward nearby model samples, thereby defining a transport direction for generated samples. In this paper, we show that drifting is more closely connected to score-based generative modeling than it may first appear, establishing a precise link to the score-matching principle underlying diffusion models. For Gaussian kernels, the population mean-shift field exactly equals the difference between the scores (i.e., the gradient-log-densities) of the Gaussian-smoothed data and model distributions. This identity follows from Tweedie's formula, which links the score of a Gaussian-smoothed density to its conditional mean, and implies that Gaussian-kernel drifting is exactly a score-matching objective on smoothed distributions. More generally, we derive an exact decomposition for radial kernels in which mean shift equals a score-based field plus a residual term. For the practical Laplace kernel, we further show theoretically and empirically that this residual is negligible in high dimension, implying that the transport field used in practice is nearly score-based. Our results reveal a structural connection to diffusion models: both methods use score-mismatch transport directions, but drifting realizes the score nonparametrically through kernel-based estimates, whereas diffusion models learn it parametrically with neural networks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑