漂移场并非保守场
Drifting Fields are not Conservative
AI总结:
本文研究漂移场是否等价于优化标量损失,发现漂移场非保守,无法表示为标量势函数的梯度。通过引入sharp kernel和sharp-normalized漂移场,实现保守性,适用于一般径向核。
摘要:
漂移模型近期因单次正向传递生成高质量样本而受到关注。训练时,它们通过跟踪向量值场(漂移场)学习推前映射。我们问这一过程是否等价于优化标量损失,发现一般情况下并非如此:漂移场非保守,无法表示为任何标量势函数的梯度。我们识别出位置依赖的归一化是导致非保守性的根源,高斯核是唯一例外的径向核。基于此,我们引入sharp kernel $k^\#$ 和保守的sharp-normalized漂移场,适用于一般径向核。所得向量场是标量势函数的梯度,可通过随机梯度下降直接优化。此外,该场形式为核密度估计的分数差,提供精确的平衡可识别性。因此,sharp归一化填补了与相关文献(如Wasserstein梯度流和去噪分数匹配)的差距,也适用于非高斯核。实验证明,sharp归一化保持了原始漂移目标的性能,表明非保守性对于高质量生成并非必需。
英文摘要:
Drifting models have recently gained attention for generating high-quality samples in a single forward pass. During training, they learn a push-forward map by following a vector-valued field, the drift field. We ask whether this procedure is equivalent to optimizing a scalar loss and find that, in general, it is not: drift fields are not conservative and cannot be written as the gradient of any scalar potential. We identify the position-dependent normalization as the source of non-conservatism, with the Gaussian kernel as the unique radial exception. Guided by this, we introduce the sharp kernel $k^\#$ and a sharp-normalized drift field that is conservative for general radial kernels. The resulting vector field is the gradient of a scalar potential that can be optimized directly using stochastic gradient descent. Moreover, the field has the form of a score difference of kernel density estimates, and gives exact equilibrium identifiability. Thus, sharp normalization closes the gap to related literature, such as Wasserstein gradient-flows and denoising score matching, also for non-Gaussian kernels. Empirically, sharp normalization preserves the performance of the original drifting objective, suggesting that the non-conservative flexibility is not required for high-quality generation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。