发表机构
Stanford University（斯坦福大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文针对扩散模型得分差估计存在的统计一致性或可扩展性问题，提出Sobolev正则化的一致可扩展估计器，在小样本场景稳定性提升，真实任务性能优于非正则化方法。
AI 中文摘要
估计两个Stein得分函数的差是生成建模中的基础问题，得分差自然出现在迁移学习中，可将预训练模型适配到新的目标分布，也出现在基于扩散模型的后训练方法（如判别器引导）中。现有得分差估计器要么缺乏统计一致性，要么难以在高维场景扩展。我们提出一种基于Sobolev正则化的统计一致且可扩展的得分差估计器，该正则化在小样本场景下对保证一致性和稳定训练起关键作用。数学上，我们建立了收敛速率$O(n^{-\frac{s-1}{d+2s-2}})$（其中$d$为维度，$s$为基础密度的光滑度），并给出了均方误差下的极小极大下界$\tilde{\Omega}(n^{-\frac{2(s-1)}{d+2s}})$。实验表明，与现有方法相比，我们的估计器在小样本场景下稳定性显著提升，在心电图信号生成的迁移学习等真实任务中，其下游分类性能大幅优于非正则化得分差估计器。
英文摘要
Estimating the difference of two Stein's score functions is a fundamental problem in generative modeling. In particular, score differences arise naturally in transfer learning, where the score difference provides the mechanism for adapting a pre-trained model to a new target distribution, and in diffusion model-based post-training methods such as discriminator guidance. Existing estimators for score differences in these settings either lack of statistical consistency or are difficult to scale up in high-dimensions. We propose a statistically consistent and scalable estimator for score differences based on Sobolev regularization, which plays a crucial role in ensuring consistency and stablizing the training in the small-sample regime. Mathematically, we establish a convergence rate of $O(n^{-\frac{s-1}{d+2s-2}})$ where $d$ is the dimension and $s$ denotes the smoothness of the underlying densities, and provide a minimax lower bound of $\tildeΩ(n^{-\frac{2(s-1)}{d+2s}})$ (in mean-squared error). Empirically, our estimator exhibits significantly improved stability in small-sample regimes compared to existing methods. We demonstrate its effectiveness on real-world tasks, including transfer learning for ECG signal generation, where it substantially outperforms non-regularized score difference estimators in downstream classification performance.
CommentsAccpeted by ICML 2026
