更快扩散指数积分器采样器的得分归一化
Score Normalization for a Faster Diffusion Exponential Integrator Sampler
AI总结：
针对DEIS采样器在采样末期得分参数化变化剧烈的问题，提出得分归一化方法（DEIS-SN），在推理时用离线高NFE生成的平均绝对值归一化得分，持续改善FID，如CIFAR-10上10 NFE时FID从6.44降至5.57。
摘要：
最近，Zhang等人提出了扩散指数积分器采样器（DEIS），用于从扩散模型中快速生成样本。该方法利用概率流常微分方程（ODE）的半线性特性，大幅减少积分误差，并在低函数评估次数（NFEs）下提高生成质量。该方法的关键在于得分函数的重参数化，它减少了在每个积分步骤中使用固定得分函数估计所带来的积分误差。原始作者使用为噪声预测训练的模型所采用的默认参数化——将得分乘以条件前向加噪分布的标准差。我们发现，尽管该得分参数化的平均绝对值在反向采样过程的大部分时间内接近常数，但在采样结束时它会迅速变化。作为一个简单的修复方案，我们提出在推理时将得分除以从离线高NFE生成中收集的该时间步先前得分估计的平均绝对值，以此重新参数化得分。我们发现，我们的得分归一化（DEIS-SN）相比原始DEIS持续改善了FID，在10个NFE下，CIFAR-10上的FID从6.44提升至5.57，LSUN-Church 64x64上的FID从5.9提升至4.95。我们的代码可在https://github.com/mtkresearch/Diffusion-DEIS-SN获取。
英文摘要：
Recently, Zhang et al. have proposed the Diffusion Exponential Integrator Sampler (DEIS) for fast generation of samples from Diffusion Models. It leverages the semi-linear nature of the probability flow ordinary differential equation (ODE) in order to greatly reduce integration error and improve generation quality at low numbers of function evaluations (NFEs). Key to this approach is the score function reparameterisation, which reduces the integration error incurred from using a fixed score function estimate over each integration step. The original authors use the default parameterisation used by models trained for noise prediction -- multiply the score by the standard deviation of the conditional forward noising distribution. We find that although the mean absolute value of this score parameterisation is close to constant for a large portion of the reverse sampling process, it changes rapidly at the end of sampling. As a simple fix, we propose to instead reparameterise the score (at inference) by dividing it by the average absolute value of previous score estimates at that time step collected from offline high NFE generations. We find that our score normalisation (DEIS-SN) consistently improves FID compared to vanilla DEIS, showing an improvement at 10 NFEs from 6.44 to 5.57 on CIFAR-10 and from 5.9 to 4.95 on LSUN-Church 64x64. Our code is available at https://github.com/mtkresearch/Diffusion-DEIS-SN
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
